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In this work we study the problem of asymptotically optimal con- 
trol of a well-known multi-class queuing network, referred to as the 
"crisscross network," in heavy traffic. We consider exponential inter- 
arrival and service times, linear holding cost and an infinite horizon 
discounted cost criterion. In a suitable parameter regime, this prob- 
lem has been studied in detail by Martins, Shreve and Soner [SIAM 
J. Control Optim. 34 (1996) 2133-2171] using viscosity solution meth- 
ods. In this work, using the pathwise solution of the Brownian control 
problem, we present an elementary and transparent treatment of the 
problem (with the identical parameter regime as in [SIAM J. Con- 
trol Optim. 34 (1996) 2133-2171]) using large deviation ideas intro- 
duced in [Arm. Appl. Probab. 10 (2000) 75-103, Ann. Appl. Probab. 
11 (2001) 608-649]. We obtain an asymptotically optimal scheduling 
policy which is of threshold type. The proof is of independent interest 
since it is one of the few results which gives the asymptotic optimality 
of a control policy for a network with a more than one-dimensional 
workload process. 

1. Introduction. Stochastic networks are ubiquitous in problems involv- 
ing manufacturing, communication and computer systems. Designing good 
controls for general multi-class networks is an important and challenging 
problem. In recent years, using tools from diffusion approximations, there 
has been a significant progress in obtaining asymptotically optimal controls 
for a broad range of stochastic networks in heavy traffic. One common ap- 
proach to the optimality question is via certain singular control problems, 
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the so-called Brownian control problems (BCP), which are obtained as "for- 
mal" heavy traffic limits of queuing networks. There are several works (e.g., 
[6, 7]) which use the optimal solution of the BCP to construct control poli- 
cies for the corresponding queuing networks. These policies seem to perform 
quite well in simulation studies, however, there are relatively few results 
showing asymptotic optimality of such policies. Recently, in [1, 4], using 
large deviation ideas, a promising technique for addressing asymptotic op- 
timality questions has been introduced. Using these techniques, the authors 
prove asymptotic optimality of a certain threshold-based scheduling policy 
for a "parallel server model." Other recent results on asymptotic optimality 
of control policies for stochastic networks are in [8, 10, 11, 12, 13]. 

In the current work we study a well-known model, often referred to as 
the "crisscross network." It has been studied in [7, 16] and in great detail 
in [11] and [10]. The network is described in detail in Section 2. The basic 
problem is the optimal sequencing of jobs in a two station- two customer 
queuing system. We consider linear holding costs and an infinite horizon 
discounted cost criterion [see (2.17)]. We believe the scheduling policy that 
we propose will also be asymptotically optimal for a finite time horizon 
cost criterion with a linear holding cost. However, for the sake of simplicity, 
we restrict our attention to the first criterion. Even though the network is 
quite simple to describe, the analysis of the control problem is rather subtle 
in that the form of an asymptotically optimal scheduling policy and the 
methods of proof seem to strongly depend on the parameter regime under 
consideration. Broadly, one can divide the study of the problem into two 
different parameter regimes: Case I. /ii/ii — /i2^2 + ^3/^2 < and Case II. 
hifii — /12/U2 + Zi3/i2 > 0, where h^s are the holding costs and fi^s are the 
asymptotic service rates (see Section 2 for precise definitions). Case I yields 
a simple threshold policy and the proof of asymptotic optimality of this 
policy is given in [16]. 

Case II is the difficult case and in [11] its analysis has been subdivided 
into 4 subcases: Case IIA, . . . , Case IID. In Case IIA, in addition to condi- 
tions of Case II, both ^2/^2 — ^3/^2 and ^2^2 — h-ifJ-i are nonnegative. The 
other three subcases (Case IIB, Case IIC, Case IID) correspond to either 
one of these two quantities being negative and the case where both are neg- 
ative. Among the four subcases, Case IIA is most amenable to analysis, 
since in this case, the "effective cost" in the reduced workload formulation, 
h(wi, W2) is monotonic in both w\ and W2 (see Remark 3.3). This mono- 
tonicity is critical in obtaining an explicit, pathwise solution to the BCP. 
Case IIA was studied in [7] with specific numerical values of the parameters 
and though the authors did not prove asymptotic optimality of the proposed 
policy, they provided results from simulation studies indicating good perfor- 
mance of the control policy. This subcase was studied in complete detail in 
[11]. The authors proposed a control policy and proved the near asymptotic 
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optimality of the policy by using quite technical machinery from viscosity so- 
lution analysis of Hamilton-Jacobi-Bellman (HJB) equations. The proposed 
policy is difficult to interpret and is not very intuitive. In addition, exten- 
sion of the methodology to more complex networks appears quite daunting. 
One of the technical obstacles in such extensions is that a general theory of 
classical solutions to HJB equations or characterization results for the value 
function via viscosity solutions of HJB equations for the limiting control 
problem are not readily available. Finally, we note that, strictly speaking, 
[11] does not obtain an asymptotically optimal policy. By "near optimality," 
it is meant that for each rj > 0, one can get a control policy (depending on rj) 
which, asymptotically, is r/-close to an asymptotically optimal strategy. In 
[10], using techniques from weak convergence theory, the authors show that 
the optimal costs for the queuing network problem can be well approximated 
by those for the optimization problem of the limiting control problem. The 
approach is quite general and powerful, but the authors do not obtain an 
actual control policy which is asymptotically (near) optimal. 

In the current work we revisit the above problem (under the same pa- 
rameter regime, namely, Case IIA) using a rather different approach intro- 
duced in the context of a "parallel server model" in [1, 4]. We present the 
BCP associated with this control problem and give the equivalent workload 
formulation. The BCP that we obtain is somewhat different from the one 
presented in [11]. Indeed, the authors there remark that their BCP is not 
well posed (see Remark 3.5 for more details on this). However, as we show 
in Section 3.1, the BCP presented in this paper has an explicit pathwise 
optimal solution. The scheduling policy we propose (see Definition 3.6) is 
directly motivated by the solution of the BCP, and therefore is easy to in- 
terpret. The scheduling policy is of threshold type and thus is quite simple 
to implement as well. In addition, our proof of the asymptotic optimality of 
the policy uses rather basic large deviation ideas which, we believe, can be 
extended to more general situations. 

All inter-arrival and service times in this work will be assumed to be 
exponentially distributed. Proofs of many of the results in this paper can 
be extended to the case where the inter-arrival and service times are i.i.d. 
with distributions that satisfy suitable large deviation estimates. Indeed, in 
the parallel server model [1], the authors prove asymptotic optimality under 
precisely such assumptions on the underlying distributions. One important 
difference in our analysis is that in one of the key results of this paper 
(Theorem 4.9), in addition to the one-dimensional large deviation estimates 
that are crucially used in the proofs of [1], we also need sample path large 
deviation estimates (Theorem 5.1) for the underlying renewal processes. For 
a more detailed discussion on extending the results of this paper to the 
nonexponential case, see Remark 5.4. 
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The paper is organized as follows. The network is described in Section 2, 
along with the formulation of the problem and assumptions. In Section 3 we 
formulate the associated BCP and the corresponding equivalent workload 
formulation. We then propose a policy that is motivated by the equivalent 
workload formulation and the solution of the BCP. In Section 4 the asymp- 
totic optimality of the proposed policy is proved through the two main 
results of the paper, Theorems 4.1 and 4.2. These results are as follows. 
Denoting the minimum cost associated with the BCP as J* and the cost 
associated with any control policy T r for the rth network as J r (T r ), we 
show in Theorem 4.1 that 

liminf j r (T r ) > J*. 

r— >oo 

In Theorem 4.2 it is shown that, in the above display, the equality is achieved 
if {T r } is the sequence of policy proposed in Definition 3.6 of Section 3, with 
an appropriate choice of threshold parameters. The key steps in the proof 
of the two main theorems are in Theorems 4.8, 4.9 and 4.11. The proofs of 
these theorems are provided in Section 5. 

2. The crisscross network. 

2.1. Queueing network model. We consider a sequence of networks in- 
dexed by r, r G § C M + , where § is a countable set: {r\,r2, ■ ■ ■} with 1 < r± < 
r2 < • ■ • and r n — > oo, as n — > oo. A sketch of the rth network is described 
in Figure 1. Description for the rth network is as follows. For i = 1,2, cus- 
tomers (or jobs) of Class i arrive according to a Poisson process with rate 
A[ and have independent exponential service times at Server 1 with param- 
eter Class 1 customers, once served by Server 1, leave the system. Class 
2 customers, after being served by Server 1, proceed to Buffer 3 and are 
redesignated as Class 3 customers. There they are served by Server 2. They 
have i.i.d. exponential service times with parameter jj,^. After service, these 
jobs exit the system. All inter-arrival and service times are assumed to be 
mutually independent and all buffers have infinite capacity. We also assume 
that the system starts empty. 

2.2. Preliminaries. Let (Q,J-,¥) be a complete probability space. All 
the random variables and stochastic processes in this paper are assumed to 
be defined on this probability space. There is no loss of generality in making 
this assumption since we work with an expected loss function (see Section 
2.4 for the definition of cost) and one can always enlarge the probability 
space to support all the processes considered in this paper. The expectation 
operation under P will be denoted by IE. 

For each positive integer m > 1, let T> m be the space of right continuous 
paths with left limits, from [0, oo) to M m , with the usual Skorohod topology 



CONTROL OF CRISSCROSS NETWORK IN HEAVY TRAFFIC 



5 



and let B(T> m ) be the corresponding Borel sigma-field. All of the continuous- 
time processes considered in this paper will have sample paths in D m . If {Z n } 
and Z are processes with paths in T> m such that Z n converges weakly to Z 
as n — > oo, we will use the notation Z n => Z to denote this. 

For each r£§ and k = 1,2, let {u k (i) : i = 1,2, . . .} be a sequence of i.i.d. 
exponential random variables with mean 1/A£ G (0,oo). We interpret u r k {i) 
to be the time (in the rth network) between the arrival of the (i — l)st 
and the rth job for Buffer k (k = 1,2). Similarly, the service times of the 
three different classes of jobs are defined as sequences of i.i.d. exponential 
variables {v T j(i):i = 1,2,...}, with mean 1/fij G (0,oo), j = 1,2,3, corre- 
sponding to the three classes. We also assume that the inter-arrival time se- 
quence {u k (i) : i = 1,2, . . .}, k = 1, 2, and the service time sequence {Vj(i) : i = 
1,2,.. .}, j = 1,2, 3, are mutually independent for each r G S. 

Define 

n 

C k (n) = J2 u k(i) for ri = 1,2, ... ,fc = 1,2; 

n 

Vj(n) = £ t£(i) forn = 1, 2, . . . , j = 1,2, 3. 

i=l 

The arrival and service processes are defined in terms of these as follows: 



A r k (t) 



sup{n>0:£fc(ra) <t}, t>0,k = l,2, 
sup{n>0:r#(ra) <t}, t>0,j = 1,2,3. 



The symbol A 7 k {t) represents the number of jobs (customers) that have ar- 
rived in Buffer k up to time t. The process Sj(t) counts the number of jobs 
that Server j could have completed if it had worked continuously during the 




Fig. 1. The rth crisscross network. 
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interval [0,t]. Note that by our assumptions on the inter-arrival and service 
times, A r k {-) and SJ(-) are Poisson processes with rates X r k and [£, respec- 
tively, for k = 1,2; j = 1, 2, 3; r G §. For notational simplicity, throughout the 
paper, we will write the limit along the sequence r n as n — > oo simply as 
"r — > oo." Also, r will always be taken to be an element of § and, thus, 
hereafter, the qualifier r G § will not be stated, explicitly. We assume that 
as r — > oo, these rates approach finite limits, namely, we make the following 
assumption. 

Assumption 2.1. There exist G (0, oo), k = 1,2, and fij G (0,oo), j = 
1, 2, 3, such that 

limA£ = A fc , \im f£ = fij, k = 1,2; j = 1,2,3. 

2.3. Scheduling control. Scheduling control for the rth network is de- 
scribed by a vector-valued service allocation process 

T r (t)^(T^t),T^t),mt)), t>0, 

where, for j = 1,2,3, 2J(i) denotes the cumulative amount of service time 
devoted to activity j (viz., working on Class j jobs by the responsible server) 
in the time interval [0, t] . The idle-time processes are defined as follows: 

iftt) = t - T[(t) - If (i), I r 2 (t) = t - TJ (t), t > 0. 

For i = 1,2, t > 0, If (f) represents the cumulative amount of time that the 
ith server has been idle in the time interval [0, t]. Recall that we assume 
that the system is initially empty. Thus, the three queue-length processes 
corresponding to the three buffers can be described as follows. For t > 0, 

f211 Ql(t) = Al(t)-SV(T[(t)), i = l,2, 

The workload process W r (-) = {(W[(t), W^(t)),t > 0} is defined as follows. 
For t > 0, 

(2 ' 2) ^ (4) .«k+#. 

^3 ^3 

The service allocation processes are required to satisfy the conditions below. 
For j = 1,2, 3, k = 1,2, r G S, 

(2.3) Tj(t) eF; t>0, 

(2.4) is a continuous nondecreasing process with Tj(0) = 0, 

(2.5) Ifc(') is a continuous nondecreasing process with 11(0) = 0, 

(2.6) Qfc(*)>0, *>0. 
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From (2.3)-(2.5) and recalling the definition of /[(•) and ij(-), we that; 
for all j = 1, 2, 3, 

(2.7) TJ is uniformly Lipschitz continuous with Lipschitz constant 1. 

Any process T r satisfying (2.3)-(2.6) will be referred to as an admissible 
control policy for the rth network. Note that we are not assuming any further 
measurability condition on T r except (2.3). 

Now we define fluid-scaled processes and diffusion-scaled processes corre- 
sponding to the processes described above. For each r G § and an admissible 
control policy T r (-) with associated queue-length process Q r (-) and idle-time 
process I r (-), define, for t > 0, 

Fluid-scaled processes. 

f r (t) = r- 2 T r {r 2 t), I r {t) = r- 2 I r (r 2 t), 

(2.8) A r {t) = r- 2 A r {r 2 t), S T ft) = r~ 2 S r \r 2 t) , 
Q r (t) = r~ 2 Q r {r 2 t), W r (t) = r~ 2 W r {r 2 t). 

Diffusion- scaled processes. 

f r (t) = r- l T r {r 2 t), I r (t) = r _1 F(r 2 i), 

(2.9) l r (t) = r- 1 (A r (r 2 t)-A r r 2 t), S r (t) = r'^S^rH) - n r r 2 t), 
Q r (t) = r _1 g r (r 2 t), W r (t) = r~ l W T {r 2 t). 

By the definitions above, we have the following identities. For all t > 0, 

(2.10) W r (t)=M r Q r (t), 



where 



M r 



V T 

1 1 



and 
(2.11) 



Ql(t) = (Af{t) - Sr(f[(t))) + r{\\t - fiT[{t)), i = l,2, 

QKt) = (s r 2 (ij(t)) - smm +r(n r 2 m) -anm- 



We also define another process X r (-), which is closely related to the scaled 
queue length process Q r (-). A formal limit of X r {-) is used in the BCP 
described in Section 3. For t > 0, let 



XT(t) = A\(t) - Sl{Tf{t)) +r(\\t - n\-t) , i = 1, 2, 
(2.12) V f \^ 

Xl{t) = Sl{T 2 (t)) - Sl(Tl{t)) + r nl-t ~ &t 

\ H2 
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Prom (2.11) and (2.12), we have the following relationships: 
(2.13) 



fri<t) = Xi(t)+ri4(-t- 

.M« 



Ql{t) = X r 3 (t) + r^(t - S£(t)) - rn r 2 (^t - f J(t)) . 

\M2 / 

We will assume that the sequence of networks is in heavy traffic. More 
precisely, we will make the following assumption. 

Assumption 2.2 (Heavy traffic assumption). We assume that the fol- 
lowing relationships hold for the limiting parameters: 

(2.14) ^ + ^ = 1, **=1, 

Ml M2 M3 

and there exist hi £ M, i = 1, 2, 3, such that lim^oo b\ = bi, where 

(2.15) vr = r(^-^\ i = l,2, bl = r( X " 



Mi Mi/ VM 3 

Under Assumptions 2.1 and 2.2, the diffusion-scaled workload process has 
the following representation: 

(2.16) W r (t) = M r X r (t) + F(t), 

where, for t > 0, F(t) = F(r 2 t)/r and J[(t) = t - T[{t) - TJ(i), F 2 {t) = t - 

rat). 

2.4. The cost function. For the rth system, we consider the expected 
infinite horizon discounted (linear) holding cost associated with the control 
T r and the corresponding normalized queue-length process Q r , given as 
follows: 



(2.17) J r (T r ) = e-*h ■ Q r (t) dtj , 



where 7 £ (0, 00) is the "discount factor" and h = (h\,h2,h%)\ hk £ (0, 00), 
k = 1,2,3, is the vector of "holding costs" for the three buffers. 

The goal is to find a sequence of admissible controls which asymptotically 
give the minimum possible cost, that is, find a sequence {T r } such that 

lim j r (T r ) =infliminf j r (f r ), 
r— >oo r— >oo 

where the infimum on the right-hand side is taken over all admissible se- 
quences {r r }. 

We will make the following assumption on the service rate and holding 
cost parameters. 
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Assumption 2.3. hifj,% - h 2 /x 2 + ^3/^2 > 0, h 2 /x 2 - ^-3^2 > and h 2 fi 2 - 
hiHi > 0. 

This parameter regime is the Case II A of [11] among the different cases 
mentioned in that paper. Case I considers the parameter regime h\^i — 
h 2 \i 2 + hzH2 < 0. This case has a simple priority policy which is shown to be 
asymptotically optimal in [16]. Case II corresponds to the complementary 
regime, namely h\^i — h 2 fi 2 + ^3/^2 > 0. In this case, for the first server, 
serving Class 1 jobs reduces immediate cost at an average rate of hifii, 
whereas serving Class 2 jobs would reduce immediate cost at an average 
rate of h 2 fj, 2 , but increases cost at an average rate of hsfj, 2 , since a job 
served from Class 2 becomes a Class 3 job. Since h±ni > h 2 fj, 2 — h^n, 2 , total 
immediate cost is reduced at a more rapid average rate by serving Class 
1 jobs. But a simple priority policy for Server 1 that requires it to work 
on Class 1 jobs, whenever Buffer 1 is nonempty, will cause starvation of 
Server 2 and is likely to cause the contents of Buffer 2 to grow without 
bound. In the Case IIA, we also assume h 2 fi 2 > h\^\ and h 2 fi 2 > h^^i 2 (or, 
simply, h 2 > /13). Here the second condition means that it is cheaper to hold 
jobs in Buffer 3 than in Buffer 2. Also, the first condition above says that 
working on Buffer 2 reduces the immediate cost at Server 1 more quickly 
than working on Buffer 1. In this work we show that, under Assumption 
2.3, a suitable threshold policy is asymptotically optimal. This policy (see 
Definition 3.6 for the precise description of the policy) keeps a sufficient 
number of jobs in Buffer 3 (so that Server 2 does not idle unnecessarily) 
and makes Server 1 work on both the associated buffers so that none of 
the buffers blow up. An example of parameters satisfying Assumption 2.3 is 
hi = h 2 = h% = 1, jUi = fx 2 = 2, fj,s = 1. In [7] the authors worked with this set 
of parameter values. 

3. Brownian control problem. We now introduce the BCP (see [3]) as- 
sociated with the crisscross network introduced above. This control problem 
is obtained by taking a formal limit of the control problems for the above 
sequence of networks. More precisely, defining 



we might expect, for "reasonable" control policies, that, as r — > 00, 



(3.1) 




t > 



(3.2) 



From the functional central limit theorem, one has that 



(3.3) 



(A r (.),S r (-))^(A(-),S(-)) 
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where A is a two-dimensional Brownian motion that starts from the origin 
and has diagonal covariance matrix, diag(Ai, A2) and S is a three-dimensional 
Brownian motion, independent of A, that starts from the origin and has di- 
agonal covariance matrix, diag(//i, fi 2 , l^ 3 )- Using (3.3), (3.2), a random time 
change theorem (Lemma 3.14.1 of [2]) and the heavy traffic condition (As- 
sumption 2.2), one has that 

(3.4) X r (-)^X(.), 

where, for t > 0, 

Xi(t) = Ai(t) - Si(T*(t)) + (M)*, i = 1,2, 
X 3 (t) = S 2 (T*{t)) - S 3 (T*(t)) + ( M3 6 3 - V2b 2 )t. 

Note that X is a three-dimensional Brownian motion that starts from origin, 
with a drift ^ 2 b 2 , ^3^3 —^2^2) and covariance matrix 

2Ai \ 
2A 2 -A 2 • 
-A 2 2A 2 / 

As stated in the beginning of Section 2.2, we can assume (by enlarging the 
probability space, if needed), without loss of generality, that A,S,X are 
defined on (O, J-,W). Thus, taking a formal limit as r — > 00 in (2.11), (2.12), 
(3.2) and (2.17), one arrives at the following Brownian control problem. 



Definition 3.1 [Brownian control problem (BCP)]. Let X(-) be as de- 
fined below (3.4). The BCP is to find an M 3 -valued measurable stochastic 
process Y(-) = (Vi(-), Y 2 (-), ^3("))> referred to as the control process, which 
minimizes 

(3.5) Ey™ e-^h-Qtydtj, 
subject to the following conditions. For all t > 0, 

0<Qi(t) = Xi(t)+/ii*i(t), 

(3.6) 0<Q 2 (t) = X 2 (t)+fi 2 Y 2 (t), 

< Q 3 (t) = X 3 (t) + fi 3 Y 3 (t) - fi 2 Y 2 (t) 



and 
(3.7) 



Ii(-) = Yi(-) + Y 2 (-) is nondecreasing and /i(0) = 0, 
^2(0 = ^3( - ) is nondecreasing and ^(0) = 0. 



We will refer to any measurable process Y(-) satisfying (3.6) and (3.7) as an 
admissible control for the BCP. 
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Remark 3.2. Our formulation of the BCP is somewhat different from 
that in Harrison (cf. [5]) in that we do not work with a weak formulation 
and we do not require the adaptedness of the control process. However, the 
diffusion control problem is not the real topic of interest here. It is used only 
to prove asymptotic optimality of our policy, namely, the result: 

lim J r (T r ) = inf lim inf J r (f r ), 
r—>oo r^oo 

where T T is our proposed policy as in Definition 3.6 and the infimum on 
the right-hand side is taken over all T r satisfying (2.3)-(2.6). In this regard, 
the formulation considered in the current work suffices. It will be seen in 
Section 3.1 that the cost in (3.5) is minimized by Y* which is adapted to 
the filtration generated by X. It follows that the infimum of the cost in 
(3.5), J* [see (3.27)], is the same as that taken over all probability spaces 
supporting a three-dimensional Brownian motion with the same drift and 
covariance matrix as X. 



3.1. Reduction to the equivalent workload formulation. Let Y(-) be an 
admissible control for the BCP and define Q via (3.6). Define the workload 
process 

W{t) = MQ{t), t>0, 



(3.8) 
where 



M 



(1- 

Hi 





1*2 
1 

1*3 



1 

f*3 I 



Thus, for t > 0, 



(3.9) 



W x {t) 
W 2 {t) 



Qi(t) , Q 2 (t) 



Q*(t) 



+ 



+ 



.^2 

Qs(t) 



M3 M3 

It is easy to check that 

(3.10) W = MX + V withV(t) = (h(t),l2(t))',t>0. 

We will now obtain a solution of the BCP using the above workload process. 
We begin by considering the following simple linear programming problem. 
Fix w±,W2 G [0, oo). The linear program (LP) problem is as follows: 

hiz\ + h 2 z 2 + h 3 z 3 

■ wi, 

(3.11) 



minimize 



Z1,Z2,Z3 



subject to 



M3 M3 
Zl,22,Z3 > 



W2, 
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A straightforward calculation using the fact that hifi\ — h 2 \x 2 + /13/U2 > 
(see Assumption 2.3) shows (cf. [11]) that the value of the LP is 

(h 2 fi 2 ~ hzn 2 )wi + (h 3 /j, 3 )w 2 , 
when /j, 3 w 2 > 

(3.12) h( Wl ,w 2 ) = { {h )wi + £s ( _ x) 

when fi 3 w 2 < fJ- 2 Wi. 
In particular, if 21,22,23 are nonnegative numbers such that tt + tt = w± 
and f 2 - + t 3 - = u; 2 , then 

(3.13) /jiZi + h 2 z 2 + h 3 z 3 > h(w 1 ,w 2 ) ■ 

Another simple calculation yields the following solution of the LP: 

z* = 0, z 2 = n 2 wi , z 3 = fi 3 w 2 - n 2 w\ , 

if fi 3 w 2 > H2W1, 

(3.14) * Hx ( s * n 
v 1 z{ = — {H2W1- H3W2), z^=fi 3 w 2 , z 3 = 0, 



A*2 



if n 3 w 2 < [i 2 w\. 



Remark 3.3. Note that from Assumption 2.3, h 2 fi 2 — h 3 /i 2 > and 
h 2 fJ, 2 — > 0. Thus, we have that h(wi,w 2 ) is a nondecreasing function 
of both w\ and w 2 . This monotonicity property is critical in obtaining a 
pathwise optimal solution to the BCP. 

We now present another control problem which, because of the mono- 
tonicity property of h, can be solved explicitly. The results of [6] show that, 
using a solution of this reduced control problem (referred to as EWF in 
Definition 3.4 below) and the solution h of the linear program in (3.11), one 
can obtain a solution of the BCP. 

Definition 3.4 [Equivalent workload formulation (EWF)]. Let X(-) 
be as defined below (3.4). The equivalent workload problem is to find an 
M 2 -valued measurable stochastic process !(•) = (Ii(-),I 2 (-)), referred to as 
the control process, which minimizes 

(3.15) EU™ e^h(W(t))dtj, 

subject to the following conditions. For all t > 

~ , . XUt) X 2 (t) ~ , , 
< Wi{t) = + -^i/jt, 

(3-16) ~ , x X 2 (t) X 3 (t) ~ ,. 

V ' 0<W 2 (t ) = ^1 + -±1+I 2 (*) and 

Ii(-),I 2 (-) are nondecreasing and 1(0) = 0. 
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From (3.16), using the minimality property of the one-dimensional Sko- 
rohod problem (see Proposition B.l of [1]), we have that if 

(3.17) (y?(t),V?(t)) = (- ml : K1( S )),- in! [ (m' 2 X (s)))' , 

\ 0<s<t 0<s<t J 

then 

(3.18) W i (t)>W*(t)±m' i X(t) + V*(t), i = l,2, 

where, for z = l,2, m! i is the ith row of the matrix M, that is, M = [mi : m 2 ]' . 
Also, from (3.18) and Remark 3.3, it follows that, for all t > 0, 

(3.19) kWi(t),W 2 (t))>h(Wm,W(t))- 

This shows that V* is a solution to the EWF. Using W* and V* , we now 
construct the solution of the BCP. The solution is motivated by the solution 
of the LP problem in (3.11), given via (3.14). Define processes 5^*(-)> i = 
1, 2, 3, as follows. For t > 0, let 



(3.20) Yi*(t) 




if (i 3 WZ(t)>[i2W?(t), 

^v 2 *(t), if ^(t) < fi 2 wnt), 

^2 



if A*aW 2 *(t) > imWM), 

(3.21) Y 2 *(t) = { 

Ml + f^-V 2 *(t), if ^(t) < n 2 Wi(t), 

1*2 l±2 

and 

(3.22) Y 3 *(t) = V 2 *(t). 

It is easy to verify that Y* is an admissible control for the BCP. Also, it 
follows from (3.20)-(3.22) and (3.16) that I* = V* . Now define Q* via (3.6), 
with Y there replaced by Y*. 

Hence, we have that if fi 3 W 2 (t) > fi 2 Wi(t), then 



(3.23) 



Q*(t) = o, 

'^\ Xl {t) + X 2 (t)+lJi2Vx{t), 



Qm=^w 2 *(t)-fi 2 w^(t) 

'^X l (t) + X z (t)+^(t)-^ 2 V*(t), 
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and if /i 3 W 2 *(t) < (MiW^(t), then 

(3.24) -m-{fym + ^vm-^vm, 

= (X 2 (t)+X 3 (t)) + fj, 3 V 2 *(t), 
QUt) = 0. 

Now we show that Y* is a solution to the BCP described in the beginning 
of this section. Note that if Y is any admissible control for the BCP and Q 
is defined via (3.6), then, from (3.13), for all t > 0, 

(3.25) h-Q(t)>h(Wi(t),W 2 (t)), 

where W\ and W 2 are defined via (3.9). In view of (3.25) and (3.19), in order 
to show that Y* is the solution of the BCP, it suffices to show that 

(3.26) h(W?(t),W 2 {t)) = h-Q*(t). 

But (3.26) is an immediate consequence of the definition of Q* [see (3.23) 
and (3.24) and the fact that z* defined via (3.14) is the solution of the linear 
program in (3.11)]. 

This proves that Y* is a solution for the BCP with the corresponding 
queue-length Q* . Let the infimum of the objective function (3.5), over all 
admissible controls, in the BCP be denoted by J*, that is, 



oc 



(3.27) J* = infE(j e _7 */i ■ Q(i) eft 
Thus, we have that 

(3.28) J* = E f jf °° e~ ll h ■ Q* (t) dt 

Remark 3.5. The BCP (Definition 3.1) presented in this work is some- 
what different from the BCP studied in Section 7 of [11]. The BCP con- 
sidered in [11] is formulated in terms of a four-dimensional control process 
which is required to have paths of bounded variation. Due to this restric- 
tion, the authors were unable to prove the existence of an optimal control 
policy for the BCP. For precise description of the control problem, we refer 
the readers to Section 7 of [11]. In the formulation considered in the current 
paper, the BCP has a three-dimensional control process which is restricted 
to have paths in 2?([0, oo);]R 3 ). Thus is our formulation, an admissible con- 
trol need not have paths of bounded variation. As seen above, the BCP in 
Definition 3.1 has an optimal solution given via (3.20)-(3.22). 
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3.2. The policy. Motivated by the solution of the BCP, we now pro- 
pose our control policy for the rth network, r£S. Fix c, G (1, oo). Define 
L r = L^ologrJ and C r = [cologrj, where Co = cIq. Since we are interested 
in asymptotic optimality, we can (and will) assume, without loss of gener- 
ality, that r > f , where f is such that for all r > f, C r — L r — 1 > 1 and 
§(C r -L r + 2)>l. 

Definition 3.6 (Control policy). The policy is as follows. No idling by 
Server 2 unless Buffer 3 is empty. The sequencing control for Server 1 is as 
follows. 

If Q5( S )-gQU*)<L r , 

serve Buffer 2 if Q r 3 (s) < C r - 1 and Q r 2 {s) / 0, 

serve Buffer 1 (when it is nonempty) if either Q^(s) > C r — 1 or 

Q r 2 (s) = o. 

If Q-( s )-g<^( s )>I/, 

,.r 

serve Buffer 1 (when it is nonempty) if either Qi(s) > ^f(C r — L r + 2) 
or Q5(a) = 0, 

serve Buffer 2 if Q\{s) < g(C r - L r + 2) and Q r 2 (s) / 0. 
Server 1 idles if both Buffer 1 and Buffer 2 are empty. 

We will refer to the constants c and as the threshold parameters of the 
control policy. It will be shown that, for a choice of c and £o large enough, 
the above policy is asymptotically optimal. One precise choice of c and Iq is 
given in Remark 4.3(a). 

One of the referees has conjectured that the above policy with £q = 
and Co replaced by a sufficiently large constant is asymptotically optimal as 
well. However, as is explained in the following paragraph, the arguments in 
the current paper crucially rely on the largeness of Iq. 

Now we provide some motivation for the policy proposed above. Note that 

S 3 wz(t) - fiww) = Q r 3 (t) - 4$;(t). 

pi 

Thus, the solution of the BCP suggests that when Qs(t) — j^Qi(t) < 0, then 
the optimal policy should try to make queue 3 empty, whereas when the 
opposite is true, queue 1 should be emptied. This is achieved in the first 
regime via the threshold C r — 1 and in the other regime via the threshold 
g(C r - L r + 2). Note that the two thresholds C r - 1 and g|(C r - L r + 2) 
approach oo as r — ► oo, however, in diffusion scaling these are negligible. 
Furthermore, (3.19), suggests that asymptotically there should be no idling 
by Server 1 unless there is no work in Buffer 1 and Buffer 2, and that 



16 



A. BUDHIRAJA AND A. P. GHOSH 



there be no idling by Server 2 unless there is no work at Buffer 2 and 
Buffer 3. The first nonidleness condition is quite easy to enforce, by saying 
that the first server works whenever there is work for it to do. However, the 
second nonidleness condition is difficult to enforce, since one can get into the 
situation where Buffer 3 is empty and so Server 2 has no immediate work to 
do but Buffer 2 is nonempty. Thus, one needs to ensure that there is always 
enough work in Buffer 3 when Buffer 2 is nonempty. This is the reason 
for the threshold L r = |/ol°g r J hi the policy. For our proof of asymptotic 
optimality, we will need that £q is sufficiently large (see Theorem 4.9). 

Remark 3.7. This policy is preemptive-resume type. For example, if 
at any time instant t Server 1 is working on jobs of Class 1 and the policy 
requires it to work on Class 2 jobs, it immediately suspends all Class 1 jobs 
and starts working on Class 2 jobs (suspended jobs if there are any, or new 
jobs). When at a later time it turns to Class 1 jobs again, it resumes working 
on the suspended Class 1 job (and spends only the excess time that it needs 
to complete the remaining part of the job, so that the total time spent on 
this job is the same as the time needed to complete this job if there was no 
interruption) . 

Remark 3.8. The above policy can be written in the following form. 
Let 

A = ju > 0: Q r 3 (u) - < Ll j> 

(3.29) B r = {u>0: Ql(u) > C* - 1 or Q r 2 (u) = 0}, 
C r = [u > 0: QKu) > 4(C r - L r + 2) or Q r 2 (u) = 
V r = {u>0:Q r l (u) + Q r 2 (u)^0}. 

Then the (sequence of) proposed policies {T r } described above in Defini- 
tion 3.6 can be described as follows. For j = 1,2,3, TJ is the absolutely 
continuous function whose derivative (defined a.e.), denoted by JJ, is given 
as follows: 

(3.30) T 2 = (I^Iflc + Uclcc)!^, 



T 3 — I{u>0: Q!;(u)>0}- 



Note that, for j = 1, 2, 3, 



IJ(t) = J*fj(s)ds 



Here 1a denotes the indicator function of a set A and A c denotes the com- 
plement of a set A. Note that T£(t) and T[(t) +T 2 (t) are both {0, 1} valued 
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and TJ(t) + TJ(i) = if and only if both Q\(t) and Q r 2 {t) are zero, and 
T^(t) = if and only if Q^(t) is zero. In other words, the policy operates in 
a "nonidling" fashion. 

4. Proof of asymptotic optimality of the proposed policy. In this section 
we will prove the asymptotic optimality of the scheduling control policy 
introduced in Definition 3.6. More precisely, we prove the following two 
results. 

Theorem 4.1. Let {T r } be any sequence of scheduling controls. Then, 
for J* as in (3.28), we have 

(4.1) liminf J r (T r ) > J*. 

r — >oo 

Theorem 4.2. There exist c,£ £ (l,oo) such that if{T r } is the sequence 
of scheduling controls described in Definition 3.6 with threshold parameters 
c and £q with £q > i, then 

(a) (W r ,I r ) =>(W*,I*) asr^ oo, 
( 42) (b) lim J r (T r ) = J*, 

r— >oo 

where J* is as in (3.28). 

Theorem 4.1 says that the asymptotic cost associated with any scheduling 
policy cannot be lower than J* defined in (3.28), while Theorem 4.2 says that 
the control described in Definition 3.6 asymptotically achieves J*, which is 
the optimal cost for the BCP. 

Remark 4.3. (a) The choice of c, I depends on various large deviation 
estimates that are obtained in Section 5. A concrete choice of c, i € (l,oo) 
is as follows: 

(* q\ ^3 = -4-niin{?7i,f/2} ) 

(4.3) /i 2 Ai 

p 2 = min{r/ 3 ,?74}, 

where 771 = ? 2 ({AU, 1/2), m = <&({l£}, 1/2), % = ^M}, min{/i 3 /2, 1}), 774 = 
?2 ({^2}: m i n {A i 2/2, 1}) and ?2( - ) is as in Corollary 5.3. Choose c,K,d,9 as 
in (5.33). Define 74 = (2d/K)9p 2 and choose 1= max{4/7 4 ,4/(6> 3 (c - 1))} + 
1. 

(b) In this paper we restrict ourselves to a discounted cost, however, sim- 
ilar results can be proved for some other cost criterion (with linear holding 
cost) as well, by suitable modifications. The key obstacle is to prove the 
uniform integrability estimates of Section 5. In particular, if the criterion is 
finite time horizon total cost, then the uniform integrability estimates are 
easy to obtain. 
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(c) In this work we will also establish (see Corollary 4.10) that, under 
the proposed policy (Definition 3.6), 

&Q$5(-)=*°- 

In [11] the authors conjectured that any optimal policy should try to get 
the queue-lengths close to the set 

{(zi,z 2 ,z 3 ) £ [0, oof :z 1 z 3 = 0}. 

Outline of the proofs. The main steps in the proof of asymptotic optimal- 
ity of the proposed policy are as follows. As a first step we show in Theorem 
4.1 that the asymptotic cost for any sequence of policies is bounded below 
by J*. The key step is proving the inequality in (4.12) and the main ingredi- 
ents in its proof are the monotonicity property described in Remark 3.3 and 
the minimality property of the Skorohod map [see (4.22)]. We next show 
that the asymptotic cost for the sequence {T r '*} is J*. The first step in this 
direction is obtaining the following convergence results for the queue-length 
and idle-time processes (see Corollary 4.10): 

0/) l {Ql(s)>di \o g r/r} dI 2 ( S ) °" 

The first two above are consequences of Theorem 4.8, while the third conver- 
gence result follows from Theorem 4.9. The latter result, along with the con- 
tinuity of the Skorohod map, is then used to show that (W r ,I r ) =>■ (W*,I*). 
We are unable to conclude from the above convergence that Q r => Q*; the 
main obstacle is showing that 

^(O I {^Ti^(.)-^w 1 r0>LVr}^^(") I {w^(O-/«aW' 1 *(O>O}' 

However, using an elementary lemma (Lemma 4.7), we show that the conver- 
gence in (4.63) holds. Since we are working with an expected cost criterion 
with an unbounded cost function, in addition to the above weak conver- 
gence results, we also need suitable uniform integrability estimates. These 
estimates are obtained in Theorem 4.11. As an immediate consequence we 
then have (4.64) and (4.66). Combining these, we obtain (4.67). This along 
with the first two convergence results in Corollary 4.10 and the uniform in- 
tegrability estimates yield (4.70). The convergence of J r '* to J* then follows 
readily. 

We begin with the following definition. Let C m be the space of continuous 
functions from [0, oo) to M m with the usual topology of uniform convergence 
on compact time intervals. We will suppress m from the notation unless 
necessary. 
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Definition 4.4 (C-tightness). A sequence of processes with paths in 
T> m (m > 1) is called C-tight if it is tight in D m and any weak limit point of 
the sequence has paths in C m almost surely. 

The following two basic lemmas are important in proving the optimality of 
the proposed policy. The proofs of the these results are similar to the proofs 
of Lemma 9.2 and Lemma 9.3 of [1]. However, for the sake of completeness, 
we have included the proofs in the Appendix. 

Lemma 4.5. Let {T r } be any sequence of scheduling policies. Then 

(4.4) {Q r (.),A r (.),S r (.),f r (.),r(.)} r& iS C-tight. 

Lemma 4.6. Let {T r } be any sequence of scheduling policies with the 
following property: 

(4.5) J({T r })= lim inf J r (T r ) < oo, 

r — >oo 

where J r (T r ) is as in (2.17). Consider a subsequence {T r } of {T r } such 
that 



(4.6) lim J r (T r ) = J({T r }). 

r'— >oo 

Then we have 

(47) (Q r '(-),A r '(-),S r '(-),f r ' (■),?■'(■)) 

1 ] ^(O,A(-),M(0,r*(-),O) asr'^oo, 

where T* is as defined in (3.1), is the constant process that is zero for all 
t > 0, X(t) = Xt, fj,(t) = fJ,t,t> 0. 

Now we are ready to prove Theorem 4.1. 

Proof of Theorem 4.1. If lim inf J r (T r ) = oo, then (4.1) holds 
trivially and so we only consider the case when lim inf J r (T r ) < oo. 
Consider a subsequence {T r } of {T r } such that 

(4.8) lim J r '(T r ')=liminfJ r (T r )<oo. 

By Lemma 4.6 and (3.3), we have that, as r 1 — > oo, 

(4.9) (A r '(-),S r '(-),T r '(-))^(A(-),S(-),T*(-)). 

Using this observation along with Lemma 3.14.1 of [2] and Assumption 2.2 
in (2.12), we have that 

(A r '(-),S r '(-),f r '(-),X r '(-)) => (A(.),S(.),f*(.),X(.)), 
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where X(-) is as defined below (3.4). Using the Skorohod representation 
theorem, we can assume, without loss of generality, that, as r' — > oo, 



(4.10) (A r (-),S r (-lX r (•)) - (A(.),S(-),f*(.),X(-)) 



a.s., 



uniformly on compacts (u.o.c). 

From the definition of the cost function J r given in (2.17) and Fatou's 
lemma, we get 

(4.11) lim j r '{T r ')>E( r e^limM(h-Q r '(t))dt). 

r'— »oo \Jo r'—*oo J 

Thus, in order to prove (4.1), it suffices to show that, for a.e. w£H and all 
t >0, 



(4.12) 



limM(h-Q r (t,to)) >h-Q*(t,uj), 



where Q*(t) are given via the formulae in (3.17), (3.23) and (3.24) in terms 
of X(-) in (4.10). Fix such that u> is in the set of probability 1 on which 

the u.o.c. convergence in (4.10) hold, and fix t > 0. Consider the following 
two cases: 



Case I: fi 3 W 2 (t,uj) > fj, 2 W?(t,u) 



Case II: ^W^t^uj) < /j, 2 Wl(t,uj), 



where W*(-),i = 1,2, are defined in terms of X in (4.10) via the relations 
(3.17) and (3.18). Note that, since we are invoking the Skorohod represen- 
tation theorem in (4.10), this W* is not the same process as in (3.18), but 
it has the same law as W* in (3.18). Once again, we retain the same symbol 
in order to simplify the notation. 

Define h r ^ = (hi' 1 , h r 2 , /ij l ) as follows: 



(4.13) hf 



h 



r,l 



h 



r.l 



h 3 fl<3 



Mi M 3 M 2 

Observe that by Assumption 2.1, as r — ► 00, 

(4.14) hf^ht, i = 1,2,3. 

From the definition of h r ' , Assumption 2.3 and (2.10), we get 

W' 1 ■ Q r (t, u) = h r { l Q\{t, u) + hfQ r 2 {t, u) + h r /Q r 3 (t, u) 



M 3 



(hiHi) 



Ql(t,u) 



(4.15) 



+ (h 3 /i 3 ) 



+ fi 2 (h 2 - h 3 

Mi 

-Q r 2 (t,ui) + Ql(t,u) 



Q r 2 (t,u) 



/'2 



/'3 



Mi 



> ~ h 3 )]W[(t,U>) + [h3fJL 3 ]WZ(t,<j) 

= a\W[(t,u:)+b\W r 2 (t,u J ), 
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where a\ = ^2(^2 — ^3) and b\ = ^3/^3. 

Next, define K r ' 2 = (hi' 2 , h 2 2 , 2 ) as follows: 



(4.16) ^ 



r,2 



r ' 

Mi 



/in 



^lMl , M3 A ^lMl 

— — + — [h 2 

M2 M3 V M2 



r,2 



fr-3M3 

Ms 



Once more, by Assumption 2.1, as r — > 00, 
(4.17) hf^h h i = 1,2,3, 

and from the definition of /i r ' 2 , Assumption 2.3 and (2.10), we get 
h r > 2 ■ Q r (t,u) = hfQl(t,uj) + h r 2 ' 2 Q r 2 (t,oj) + fc^OS&w) 



Ql(t,u) + Q r 2 (t,u) 



(4.18) 



+ 



M3 

M2M3 



Mi M2 
[(/12M2 - h!fii)Q r 2 (t,Lo) + (h 3 fi 2 )Ql(t,u>) 



>(hiin)W[(t,u) + 



111 

M2 



(/12M2 - him) 



= a* 2 Wl(t,uj)+blW r 2 (t,uj), 

where a 2 = hifii and &?j = ^3(^2^2 — hi/ii)/ /j, 2 . 
Thus, defining 



fc(w, t) 



1, if/i 3 l^'(t,a;)>^2W 1 *(t,w), 

2, if /i 3 l^(t,w)<At3W?(t,w), 



we have that 

(4.19) h r « u >* ■ Q r (t,u>) > at {Utt ffi(t,u)+b* k{Utt) WZ(t,u). 

Note that, from Assumption 2.3, we have that a*,b* are nonnegative. Since 
W r (t) is nonnegative for all t > and I[,I 2 are nondecreasing and start 
from zero, we have from (2.16) and the minimality of the solution of the 
Skorohod problem (see Proposition B.l in [1]) that, for all t > 0, 

(4.20) il(t) > - ini UM^Xl(s) + M^ 2 X r 2 (s) + M&X^a)), i = 1,2. 

0<s<t 

For xGV 1 with x(0) = 0, define T(x) G P 1 as 

(4.21) r(x)(t) = x(t)- inf x(s), t>0. 

0<s<t 

Then (4.20) implies that 

tf7(t) > T(Ml x Xl(.) + Ml 2 Xl(-) + M^Xl(.))(t) 

for alH>0,i = l,2. 



(4.22) 
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Now we prove the inequality in (4.12). If the left-hand side of (4.12) is infi- 
nite, then the inequality holds trivially. Otherwise, get a further subsequence 
indexed by r" (which may depend on u>,t) such that 

(4.23) lim h-Q r "(t,oj) = limmfh-Q r '(t,uj) < oo. 

r"— *oo r'— »oo 

Notice that from (4.23), since hi > 0,Ql"(t, uS) > 0, for i = 1,2,3, we have 
{Ql (t,u)} is a bounded sequence as r" — > oo for i = 1,2,3. From (4.14) 
and (4.17), we have 

(4.24) lim (hl" MuJ ' t] -h i )Q r i "(t,uj)=0 fori = 1,2,3. 

r"— >oo 

Using the above equality along with (4.19), (4.22) and the nonnegativity of 
a*,b*; i = 1,2, we have 

lim (h-Q r "(t,uj)) 

= lim (h r "' k ^-Q r "(t,u)) 



>limsup(a^ (W)t) WT (t,u) + b* k(Utt) WS (t,*)) 
" l 



r"— >oo 

> hmsup[4 ( ^ ) r(M[;X["(.,u;) + M& + M&Xf (; U ))(t) 

4.25 

+ bt { ^ t) T(M£xr (-, u) + M r 22 X r 2 (-, W ) + M£ (,u,))(i)] 
= < W) r(M 11 A > 1 (-,w) + M 12 X 2 (.,w) + M 13 X 3 (-,u))(t) 
+ bl { ^ t) T(M 21 X 1 (-,Lo) + M 22 X 2 (-,u) + M 23 X 3 (-,uj))(t) 

= Ofc( W ,t)^l + &fc( W ,t)W2 

= h-Q*(t,u), 

where the fifth line follows from (4.10) and the continuity of the one-dimensional 
Skorohod map T(-) on TJ 1 and recalling that M r — > M as r — ► oo. The last 
equality in (4.25) follows from the definition of a*,b*,i = 1,2, definition of 
fe(-, •) and (3.23)-(3.24). This proves (4.12) and the result follows. □ 

We now proceed to the proof of Theorem 4.2. We begin with the following 
elementary result. 

Lemma 4.7. Let {f r },{gr} be sequences of functions in T> , and f,g be 
functions in C 1 such that f r — > f,g r — > g in T> 1 as r — > oo. Suppose that 

roo 

(4.26) J e-n {m]=0} dt = 0. 

Let {e r } be a sequence of nonnegative numbers such that e r — ► as r — > oo. 
Then, for all T > 0, the following hold: 

(4.27) e-^f r (t)I {grm£r} dt - jT e-^/(*)I{ ff (t)>0} * m r oo, 
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(4.28) e-^f r (t)I {gr(t)<£r} dt - £ e-^f(t)I {gmo} dt as r - oo. 

Proof. Let fi be a finite measure denned on (R+, S(R_|_)) via the fol- 
lowing relation: 

(4.29) dfi(t) = e" 7 * dt. 
It follows from (4.26) that 

(4.30) g(t)^0 a.e. t[/j]. 
To prove (4.27), we will show that 

/ Mt)I{ gr (t)> Er } d(j,(t) - J /(i)I {fl(t) > }rf/x(i) ^0 asr^oo. 

(4.31) 

We can bound the left-hand side of (4.31) by 

(4.32) [ T \f r (t) - f(t)\ dn(t) + f T \f(t)\ ■ \I {gr{t )>e r} ~ I {g{t) > 0} \d^t). 

^ " 

Since / is continuous and f r — > f in V 1 , we have sup 0<t<T \f r (t) — f(t)\ — > 0. 
This shows that the first term in (4.32) converges to zero as r — > oo. 
For the second term in (4.32), it is enough to show the following: 

(4.33) l {g r {i)>e r } l {g{t)>o} as r ^ oo for a.e. t\pi\. 

But (4.33) is an immediate consequence of (4.30) and the fact that, since g 
is continuous, 

(4.34) l{ gr (t)>e r } I { 3 (*)>0} as r -> oo for all t such that g(t) / 0. 

This proves (4.33) and completes the proof of (4.27). The proof of (4.28) is 
similar. 

□ 

The following three theorems, the proofs of which are deferred to Section 
5, are key to the proof of Theorem 4.2. 

Let {T r } be the sequence of scheduling controls described in Definition 
3.6. Let k be a positive constant satisfying 

(4.35) K>max / 2 ^1,4,-^— , ^g£- ,fl 3 \, 

I A*2 (c-1) Mc-l) J 

where #3 is as in Remark 4.3. For r G S, t > 0, define an event S(r,t) as 
follows: 

£(r,t) = { sup Q r 3 (s)I { Q, w __ { ^ /K) Q I(s)<Lr/r} > 

(4.36) 

0<s<t 



r 

K{C r - L r + 1 
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Note that the event £(r,t) depends on parameters c and £q, however, this 
dependence is suppressed in the notation. 



Theorem 4.8. Let {T r } be the sequence of scheduling controls described 
in Definition 3.6 with threshold parameters c and £q. Let 9% be as in Re- 
mark 4.3(a). Then there exist constants Oi G (0, oo), i = 1,2, and ro > 1 
such that for all t G [0, oo), r > ro; £q E (1, oo) and c > 1 + ^, 

(4.37) ¥{£{r,t))<6 l {l + rH 2 ){e- e2rH + r- e ^ c - 1 ^). 

Theorem 4.9. There exists c£ [1 + ^,oo) such that, if {T r } is the 
sequence of scheduling controls described in Definition 3.6 with threshold 
parameters c and some £q G (l,oo), then there exist constants 7$ > 0,i = 
1, . . . , 4, n > 1, d G (0, 00) suc/i t/ia£, /or all r>r\, t G [0, 00), w;e /iaue 

IP / I/or/ w,« , / t. din (s) 7^0 

(4.38) U[0,t) W2W>^ologr/r} 2V If _ 

< 7i(l + r 2 t)e" 72r2i + 73(1 + r 2 i) 2 r-^°. 

Proofs of Theorems 4.8 and 4.9 will be given in Section 5. An immediate 
corollary of the above theorems is the following. 



Corollary 4.10. Let c and the scheduling sequence {T r } be as in The- 
orem 4.9. Suppose that £g (0, 00) is large enough so that #3(0— l)i>4 
and 74^ > 4. Then, for each fixed t > 0, for all £q > £, the probabilities 
(4.37) and (4.38) tend to zero as r— >oo. This, in particular, implies that, 
as r — > 00, 

Ql(-)^Q-(-)-^y^)Ql(-)>^/r} °. 
(4.39) GSOl^fO-C^/^^^Lr/r}^ ' 

L , I {Q5(«)>^ologr/r} d ^ («) 0. 



Using the third convergence result above, we will obtain in Theorem 4.2(a) 
that (W r ,L r ) =>■ (W'*, I*) as r — > oo. However, we are unable to show that 
Q r =>• Q* as r — ► oo. Nevertheless, as will be seen in the proof of Theorem 4.8 
below, the weak convergence results in Corollary 4.10 with suitable uniform 
integrability estimates (Theorem 4.11, see Remark 4.12) will suffice for the 
proof of asymptotic optimality of the proposed policy. 



Theorem 4.11. Suppose that c is as obtained through Theorem 4.9 and 
£q satisfies the conditions in Corollary 4.10. Let {T r } be the sequence of 
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scheduling controls described in Definition 3.6 with threshold parameters c 
and £q. Then the following hold: 



POD 

(4.40) lim sup / e~ 7 *E 



sup W[(s) 

0<s<t 



dt < oo, i = 1, 2. 



Also, 



(4.41) lim sup lim sup / e 7< E 

T^oo r^oo JT 



sup W[(s) 

0<s<t 



dt = 0, i = l,2. 



Proof of Theorem 4.11 will be given in Section 5. 

Remark 4.12. Note that (4.40) implies that W((-), i = 1,2, are uni- 
formly integrable (u.i.) with respect to the product measure Fx/j, where /j, 
is as defined in (4.29). Also, note that (4.41), in particular, implies that 

POO 

(4.42) lim sup lim sup / e" 7 *E(W i r (t)) dt = 0, i = 1,2. 

Lemma 4.6 showed that if {T r } is any sequence of admissible controls 
which gives a finite cost asymptotically, that is, (4.5) is satisfied, then T r =>- 
T* . For the sequence of scheduling controls in Definition 3.6, we do not know 
a priori that (4.5) is satisfied. In view of that, we prove the following lemma. 

Lemma 4.13. Let {T r } be the sequence of scheduling controls described 
in Definition 3.6 with threshold parameters c and t§. Suppose that c is as 
chosen in Theorem 4.9 and lo satisfies the conditions in Corollary 4.10. 
Then 

(4.43) T r ^T*, 
where T* is as defined in (3.1). 

PROOF. From (2.16), we have that, for s > 0, 
(4-44) ftr {a) = Ej& + B& 

Now, the left-hand side of (4.44) is nonnegative, and by definition of the 
proposed scheduling policy (Definition 3.6), /[ is nondecreasing, starts from 
zero and increases only when both Q\ and Cf 2 are zero, or, in other words, in 
view of (2.10), only when the left-hand side of (4.44) is zero. Let T(-) be the 
Skorohod map defined in (4.21). From a well-known characterization of the 
solution of a one-dimensional Skorohod problem (see [1], Proposition B.l), 
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we have from (4.44) that (4.20) and (4.22) hold with inequalities replaced 
by equalities. In particular, 

(4-45) W[(s)=r(^P- + ^P-)(s) 



Mi M2 



and 



(4.46) ,«., = - hf | + *K«> 



0<u<s\ fj,[ /J,' 2 

Next recall that 

(a a^7\ \xtt i \ QKs) Qlis) X r 2 {s) X r 3 (s) fr . . 

(4.47) W^(s) = ^^ + ^ F 1 = ^r 1 + ^jr 1 +r 2 {s), s>0. 

M3 M3 M3 M3 

From the second equality in (4.47), we get that, for s > 0, 

fjr L {Ql{s)>2d log r/r} ^ 



XK(s) XUs 



(4.48) V Ms + »3 1{Q 



Q r 2i s ) x 



[0,s 



I {Q-(«)>d logr/r}^2(«)) 



+ J[0,s] I {Q5(«X^logr/r} d/ 2 

Once more, from the definition of the scheduling policy (in Definition 3.6), 
the left-hand side of the equation in (4.48) is nonnegative and the last 
term on the right-hand side of (4.48) is nondecreasing, starts from zero 
and increases only when the left-hand side is zero. Also, note that since the 
paths of Q^') are piecewise constant, the processes 



/4 1 {Q r 2 (-)>2d logr/r} 

lg the characterizing 

property of the one-dimensional Skorohod map and (4.47)-(4.48), we obtain 



and ^ I{Qr(.)<2rf iogr/r} nave paths in T> 1 . Thus, using the characterizing 



V M3 M3 M3 {O5(0<2*logr/r} 

, <&(*) T 

+ ^ 1 {Q^(t)<2d logr/r}' 
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3a] I {(3S(u)<d Iogr/r} dr ^ U ) 
(4 50) ' -- inf (*M + *M - 9Ml - 



+ io s] I {Q2(«)>^logr/r} («)J • 



'[0,s] 

Using the fact that T(-) is Lipschitz continuous with constant 2 along with 
(4.45) and (4.49), we have the following: 



(4.51) 



and 



bupWTW< -r(^ + ^%) 

o<s<t o<s<t r \ n{ n' 2 J 

< 2 -L sup |X[( S )| + 2-L sup \x^ s ) 
r/J-i o<s<t rfi r 2 < s < t 



sup W r 2 {s) 

0<s<t 



+ / 0i . ] I {^(«)>d logr-/r}^2(n))( S ) 



(4.52) + SUP — IrAr 



< s < t r Jij, {Q5( S )<2dologr/r} 

<2^ sup |X£( S )| + 2^L sup \Xl{s)\ 
r\i 2 o<s<t rfj,' 3 o< s <t 

r S < U s < 4 MS 1 {OS(-)<2dologr/r} 

From (3.3), (2.12) and Assumption 2.1, it follows that, for z= 1,2,3, 
(4.53) — - sup |Xf(s)| — >0 in probability, as r — > oo. 

Tf4 0<s<t 

Also, note that 

(454) ^-^-^W^logr/r} ^ 

Now using (4.53), (4.54) and Corollary 4.10 in (4.51) and (4.52), we get that 
(4.55) W{=>0, i = l,2. 
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This immediately yields that 

(4.56) Qi^O, i = l,2,3. 
Also, from (3.3), we have that 

(4.57) %(-)=>Q, ^(2J(.))=^0, i = l,2,j = l,2,3. 
Using (4.56) and (4.57), it follows from (2.11) that, for all t > 0, 



(4.58) 



Xlt-^f[(t)^0, i = l,2, 



The result follows on combining (4.58) with Assumption 2.1 and (2.14) of 
Assumption 2.2. □ 

We now come to the proof of the main result of this section. 

Proof of Theorem 4.2. Suppose that c is as obtained from Theorem 
4.9 and £ = maxj ^^^ , — }, where #j,7i are as in Theorems 4.8 and 4.9, 
respectively. Henceforth, the sequence {T r } will have threshold parameters 
c and Iq, with £q £ (I, oo). 

From Lemma 4.13 and (3.3), we have that 

(4.59) X\ ^Xi as r-KX>,i = 1,2, 3, 

where X is as defined below (3.4). From this, Corollary 4.10, Assumption 2.1 
and alternative expressions for W[, If, for i = 1,2, in (4.45), (4.46), (4.49) 
and (4.50), it follows that 

(4.60) {W r J r )^(W*J*) asr^oo. 

We have also used (4.54) and continuity of T(-) in obtaining (4.60). This 
proves part (a) of the theorem. 

For part (b), first we observe that from Theorem 4.11 (see Remark 4.12) 
and part (a) of this theorem, it follows that 

/•OO /"OO 

(4.61) / e" rt E(Wl(t)) dt -> / e^^W*^)) dt, z = 1,2. 
Jo Jo 

Next observe that the reflected Brownian motions W* and WJ satisfy, for 
every t > 0, P(n 3 W%(t) = n 2 W?(t)) = 0. Using this fact and Fubini's theo- 
rem, it follows that 

roo 

( 4 -62) / e-n { ^ {ty ^ m=0} dt = a.s. [P]. 
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From Lemma 4.7 [see (4.27)], (4.60), (4.62) and the fact that ^ = Iq^SJL 
decreases to as r — > oo, we have, for all T > 0, 

r T 

e 7t Wi (*)I{^r^r( t )_ M r^-T-( t )> L r/ r } dt 

rT 



(4.63) 







J e ^*^\^W*{t)-^ 2 W*{t)>0} dt 



in distribution. Using uniform integrability of W{ (see Remark 4.12), we can 
conclude from (4.63) that, for all T > 0, 
rT 

-/ o T e-^E(^(t)I {M3 ^ (t) _ M2 ^ ( ^ 0} )^. 
From (4.42), for i = 1, and (4.64), simple calculations show that 



(4.64) 



(4.65) 



/■oo 

/ e 7 E (W /r i'(*)I{ / ,r^r (t) _ At r l yr (f) > Lr/r} )dt 
/■OO 



Similarly, using (4.28) of Lemma 4.7 and (4.42) for i = 2, it can be shown 
that 



(4.66) 



J e 7 E(W2 W^H^W-^^t^L'/r})^ 

/■OO 

- y o e -^(wi(t)i {M3 ^ (t) _, 2 ^ (t) < 0} )di 

From (4.65), (4.66), (3.23)-(3.24) and Assumption 2.1, it follows that 

roc 

( 4 -67) + / e- 7t E(^^(t)I {M ^, (4) _ M ^, (t)<Lr/r} )dt 

roc 

-► / e _7 *E(Q2(t))dt. 
Jo 

Now using (2.10), the left-hand side of (4.67) can be written as 

roo 

/ e- 7t E(Q^(t))dt 



r/Y 



From the uniform integrability of W[ given in Remark 4.12 and recalling 
that fj>l — > fj,i,i = 1,2, we have that, for j = 1,2,3, are uniformly inte- 
grable (with respect to the measure P x /i). Combining this observation 
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with Corollary 4.10, it follows that the last two terms of (4.68) tend to zero. 
This, in view of (4.67) and Assumption 2.1, implies that 



(4.69) 



as r — ► oo. 



-^E(Q r 2 (t)) dt -> / e^E(Q^t)) dt 
Jo 

Now using (4.69) and (4.61) in (2.10) and (3.8), it follows immediately that 

i 

e- 7 *E(Q[(t))dt 



(4.70) 



~ 7t E(Q*(i))cft aar->oo,t = l,3. 



Finally, combining the above two displays with the definition of the cost 
function and the representation of J* in (3.28), it follows that 



3 / noo \ 

(4.71) J r (T r ) = J2^J e~^hiQl{t)dtj ^J* 



as r — > oo. 



This completes the proof of the theorem. □ 

5. Proofs of Theorems 4.8, 4.9 and 4.11. We begin with the following 
standard large deviations estimate for Poisson processes. This estimate will 
be used in many of the arguments in this section. For a proof we refer the 
reader to [9] or Theorem 5.3 of [15]. 

Theorem 5.1 (Kurtz [9]). Let N\{-) be a Poisson process with rate 
A > 0. Then for allO < A < A < oo, there exists a C\ G (0, oo) and a function 
C*2 : (0, oo) — ► (0, oo) such that, for all a> 0, e > 0, 

N x (at) 



(5.1) 



sup 

AG[A,A] 



sup 

0<t<l 



Af 



a 



>£) <C ie - aC2{£) . 



An immediate corollary of the above result is the following. 

Corollary 5.2. Let {N r (-)} rG § be a sequence of Poisson processes with 
rates g r such that g' r — > g E (0, oo) as r — > oo. Then there exists aC\ G (0, oo) 
and a function C2 ■ (0, 00) — ► (0,oo) such that, for all e' > 0, 6 G (0, l),c* > 
0, r G S, we have 

N r (s) - g r s 



(5.2) 



sup 

9c* log r<s<c* logr 



The above corollary follows from some straightforward calculations on set- 
ting a = c* logr and e = Be' in Theorem 5.1. Another important consequence 
of Theorem 5.1 is the following "terminal time" estimate. 
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COROLLARY 5.3. Let {iV r (-)} re § be as in Corollary 5.2. Let e > be ar- 
bitrary. Then there exist q = Q({Q r },s) £ (0, oo), i = 1, 2, and T\ = n({^ r }, e) £ 
(0, oo) such that, for all r>r\, 

(5.3) ¥(N r {t) > (g r + e)t or N r (t) < {g r - e)t) < ae _rat Vie[0,oo). 

Remark 5.4. The large deviation estimates in the above three results 
are used in the proofs of Theorems 4.8 and 4.9, which, in turn, are key 
to the proof of Theorem 4.2. Proof of Theorem 4.1 does not rely on any 
large deviation estimates and can be extended in a straightforward manner 
to the case of more general inter-arrival and service time distributions for 
{u r k (i) , Vj (i) : k = 1, 2;j = 1, 2, 3; i = 1,2, . . .} such that the corresponding re- 
newal processes, A^(-),Sj(-), satisfy a functional central limit result similar 
to (3.3). However, in order to extend Theorem 4.2 to a general renewal pro- 
cess setting, more stringent moment conditions on the above distributions 
are needed. Proof of Theorem 4.8 uses the one-dimensional large deviation 
estimate in Corollary 5.3. Corresponding results for a general renewal pro- 
cess are well known and indeed were used by the authors in [1] to prove the 
asymptotic optimality of their policy. Under precisely the assumptions of [1] 
on the underlying renewal process (Assumption 3.3 of that paper), one can 
extend the proof of Theorem 4.8 to a nonexponential setting. Note, however, 
that the proof will need to be modified to account for the non-Markovity 
by using multi-parameter nitrations and stopping times and using Lemma 
7.6 of [1] in place of the strong Markov property. These modifications are 
fairly straightforward. Proof of Theorem 4.9 crucially relies on Corollary 5.2, 
which is a statement on the sample path large deviations of the underlying 
renewal process. We conjecture that using Theorem 3.1 of [14], one can ex- 
tend the proof of Theorem 4.9 to a larger class of renewal processes which 
satisfy suitable exponential moment conditions. 

Now, we proceed to the proof of Theorem 4.8. We begin by defining the 
following family of stopping times with respect to the filtration {J-J}t>o, 
where T r t = cr{QJ(s) : < s < t,j = 1,2,3}. For re § and n = 1,2,..., define 

T o=°< 

(5.4) r ^- 1 = hlf {* > T ^-2\Ql(t) - ^Q[(t) > 1/ j, 

r r 2n = infji > r^lQKt) - ^Q\{t) < L r and Q$(t) < C r - l}. 

From the form of the scheduling policy in Definition 3.6, it follows that 
Qs( T 2n-2) < C r — 1. Thus, Q T 3 (s) starts from below C r — 1 on [T| n _ 2 , t^^), 
and whenever the queue-length crosses (C r — 1), Server 1 stops serving 
Buffer 2, causing Q^(s) to decrease monotonically. Thus, we have that 

(5.5) Ql(s)<C r forall S e[r 2 V 2 ,rL-i],n = l,2,.... 
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(5.6) 



Proof of Theorem 4.8. Recalling the definition of the diffusion- 
scaled processes Qi(s), Qs(s), we can rewrite S(r,t) from (4.36) as 

I sup Q r 3 (s)I{Q5( s )-^/^)Qr( s )<L-} > k(C v -L r + 1) 

l0<s<r 2 t 

U \ SUp Q\ (s)I { Qr (s) _ (M r /jU r )Q r (s) > L r } > /<C r - L F + 1) \ . 
l0<s<r 2 t ) 

Let 

(5.7) n r = [(A? + X r 2 + 2)r 2 t] + 1. 

Note that every r^_ x [k = 1, 2, . . .) corresponds to one up-crossing of Q^(s) — 

..r 

^Qi(s) from below L r to the threshold level L r or above. Each such up- 
crossing either requires at least one service of a Class 1 job, which, in turn, 
implies at least 1 arrival of a Class 1 job, or it requires one service of a 
Class 2 job, which implies 1 arrival of a Class 2 job has occurred. Thus, the 
number of T 2k _ x in the interval [0, r 2 t] is bounded above by A\ (r 2 t) + A 2 (r 2 t) . 
Therefore, 

P(r 2 V-i < r 2 t) < P(^(r 2 t) + A r 2 (r 2 t) > rf) 

< P(A£(r 2 t) + A r 2 (r 2 t) > (X{ + X r 2 + 2)r 2 t) 

< P(^(r 2 t) > (A; + l)r 2 t) + F(A r 2 (r 2 t) > (A 2 + l)r 2 t) 

< Kl e- K2r2t , 

(5.8) 

for all r>h = maxfatfA;}, l),n({X r 2 }, 1)}, where Kl = &({AJ}, l) + a({AS}, 1), 
k 2 = min{? 2 ({Ai}, 1),? 2 ({A 2 }, 1)}, and n(-), Ci(-)> ft(') are as in Corollary 
5.3. 

Using (5.8) and the representation for S(r,t) in (5.6), we have that, for r 
sufficiently large, 

H£(r,t)) 

<P(r 2 V-i<r- 2 i) 



(5.9) 



+ P(r 2 V_i>r 2 t, 



" 



sup Q3( s ) i {QS(s)-(m5/mI)QIW<l-} > K (° r - Lr + x ) f 

U <^ SUP Q^(s)I { Qr {s) „ ( ^ /At p Q r (s) > Lr} >K(C r -L r +l)l 



< Kl e- K2r2t 



n=l 
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(5.10) 

T 

and Q r 3 (s) - ^Ql(s) < L r for some s G [r 2 r n _i^2n A O 2 *)] 
Mi 

+ £ P(r^ < r 2 t, QJ(a) > K (C r - L* + 1) 

n=l ^ 

(5.11) andQ5( S )-(^M( S )>L r 

for some s G [r^.^r^ A (r 2 i)] 

+ 2 £ P(d < A <E00 > ^"(C r - L r + 1) 

( 5 - 12 ) n=l 

for some s G [r 2 r n _i,T2 n A (r 2 t)]), 
where n" = min{K, f^-} and the display in (5.12) is a consequence of the 
fact that, for each summand in (5.10), Q 3 (s) — j^Qi(s) < L r . Combining this 
with the condition Q r 3 (s) > k(C t - L r + 1) gives that Q\ (s) > j£ [n(C r - L r + 

1) - L r ] = f g(C r - L r + 1) + g[f (C r -L r + 1) - 1/] > f g(C r - L r + 1), 
using (4.35) and the fact that c > 1 + 4/03. Choosing r to be sufficiently large, 
so that g > gives that, for such r, QJ(a) > n"(C r - L r + 1). The sum 

(5.10) follows from (5.8), the fact that (4.35) implies C r < n{C r - L r + 1) 
and (5.5). The third term (5.11) is obtained using the fact that the indicator 
restricts us to the values of s for which Q 3 (s) — -p-QJ(s) > L r , which happens 

only for sG [r 2 r n _i, r T 2n ). 

Note that by our choice of k [see (4.35)], we have that 

(5.13) K">max(2 — ,■ 



fi 2 c-1 

For fixed r G § and n > 1, define a sequence of "intermediate" stopping 
times within [tJ^i, t^) as follows. For m = 1, 2, . . . , 



% — r 2n-l! 



r,n 

= mm 



(5.14) and Q r 3 (s) > (C r - 1' 



OR ( Ql(s) - ^-jQi(s) > L r 
Mi 
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r,n 
V2m 



mm 



r,n 

s > V2m-1 



and Q\(s) > ^(C r 
Q3(*)-4^(*)>L r 



L r + 2) 



/'i 



Mi 



and Q^(s)<^(C r -L r + 2) 
M2 



Now, we estimate how many rf' n, s there can be in [t^-i,"^ A (r 2 i)]. Let n r 
be as in (5.7). Note that ?72n r -i — r ^ implies that there are at least a total 
of n r arrivals in Class 1 and Class 2 together in [0,r 2 i]. Using an argument 
similar to that used in obtaining (5.8), we have that 



(5.15) < rl n A irh)) < me-**, 

where are as in (5.8). Now, each summand in (5.12) can be split over the 
sub-intervals formed by the ^'"'s spanning \j2 n -\-,T r 2n A (r 2 t)] as 

Hrk-i < r 2 t,Q{(s) > K"(C r - L r + 1) for some s G [rLi,4 A (r 2 t)]) 

^Qi«-i)<4(C r -L r + 2) + l, 

Q r l {s)>K"{C* -L r + 1) 

for some s G A (r 2 t)] 



(5.16) 



n' 

< Kl e- K2r2t + ^ 1 

7 f i = 1 



r,n 
r ?2m- 



Note that the terms corresponding to s G [%m-2' 7 72m-i A (f 2 *)] do not con- 
tribute to the sum (for large values of r), as the corresponding probabilities 

are zero. This is because, for s G [??2m-2) 7 ?2m-i)> Qi( s ) < ^H^ 1 " — L r + 2), 
which in view of (5.13), implies, for large r, < n"{C r — L r + 1) for all 

s e [ 7 ?2m-2' rfim-x ^ ( r2 0)- Note that in all these calculations definition of 
is used. This observation also provides the bound that if %m-i — r2 ^> 
we have Qi(%m-i) < ^(^ r — L r + 2) + 1, which is used in (5.16). Combin- 
ing (5.12) and (5.16), we have, for sufficiently large r (choice of r does not 
depend on t), 



¥(£(r,t)) 



+ 2E 



(5.17) 



n=l 



m=l 



r,n 
V2m- 



2m— 1 j 



<^i(C r 



L r + 2) + l, 
L r + 1) 

for some s G [^m-i^m A (r 2 t)] 



M 2 

q;(s)>«"(c 
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Let A 1 ^ 1 denote the event in the (m, n)th summand in the last term of (5.17). 
On A 7 ^ 1 , our policy requires Server 1 to work continuously on Buffer 1 for 
all s G [%m-i'%m A ( r2 t)) and at the beginning of the interval, Q\ is at 
most -pr(C r — L r + 2) + 1. Also note that, for r sufficiently large, Buffer 1 
cannot be empty during this period. Indeed, for s E [v^m-i^V^m A (r 2 t)), we 
have s < rf^ < r 2n and so by definition of r^, either Qjj(£) — jjP-Qi(t) > L r 

or Q^i) > C r — 1 has to hold, and since r/ 2 ^ > ??2m-i' by definition of i)^, 
one of the following things must be true: 

(5.18) Ql( s )-^Ql( s ) < L r and Q r 3 (s)> C r -1, 

Mi 

(5.19) Q-( s )-4QI(*)>L r and Ql(s) > 4(C r - L r + 2). 

Mi M2 

It is easy to see that in either case Qi(s) > 0, using the fact that C r — L r > 1 
for r sufficiently large. 

Let, for r G §, Q r (-) denote an M/M/l queue-length, with arrivals at rate 
X[ and service times at rate n\. Define the stopping time 

P r = infjs > 0: Q r (s) < ^(C r - L r - 1) j. 

Then using the memoryless property of the exponential distribution and 
the form of the scheduling policy, it follows that each summand in (5.17) is 
bounded by 



(5.20) 



Q r (s) > K "(C r - L r + 1) for some s G [0,(3 r ], 

S r (0)<4(C r -L r + 2) + i 

M2 



For e 2 > 0, define s r = [^(C r - L r - 1) - 2]/2(AT +e 2 ). Also define 

(5.21) ^ = {A r (s r ) < (X[ + e 2 )s r , S r (s r ) > - e 2 )^}, 

where ^4 r and S 17- are the arrival and service processes of the M/M/l queue. 
From Corollary 5.3, it follows that 

(5.22) F({y r } c ) < K3 e- K4sr 

for all r > f 2 = max{ri({A^}, e 2 ), ri({/4}, e 2 )}, where k 3 = s 1 ({X r 1 },e 2 ) + 
?i({^},e 2 ), k 4 = min{? 2 ({A r 1 },£2),^2({A i i} ; £ 2)} and n(-), ?i(-), ^C) are as 
in Corollary 5.3. 

Let A r denote the event in (5.20). First, we argue that on y r n A r , we 
have s r > /3 r . To show this, we argue by contradiction. Note that if s r < f3 r , 
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then, since Q r (s) 7^ for all s < (3 r , we have 
Q r (s r ) = Q r (0) + A r (s r )-S r (s r ) 

< 4(C r - L r + 2) + 1 + (\[ - fj,[ + 2e 2 )s r . 
M2 

Choose r3 large enough and £2 > small enough, such that (A£ — n\ + 2e2) < 
(Al ~ Ml) , and (m ~ Al) s r > 1 + 3j|, for all r > r 3 . Then for r > r 3 , 

(5.23) G»V) - 4(C r - L r - 1) < 1 + 34 " f ^=^V < 0, 

which, by definition of /3 r , is a contradiction. This implies that for large 
enough r, on the set y r n .A r , s r > (3 r . Thus, on this set, for all < s < (3 r 
and large enough r, 

Q r (s) = Q r (0)+A r (s)-S r (s) 

< Q r (0)+A r (s r ) 

< 4(C r - L r + 2) + 1 + (A? + e 2 )s r 



f'2 



<M(c r -L r + l) 
<K"(C r -L r + l), 

where the last inequality follows on recalling that k > 2 — and 2— > tt-t 

for large r. This proves that P(3^ r D A r ) =0. Therefore, using (5.20), we 
have that each summand in the last term of (5.17) is bounded by P({3^ r } c ). 
Hence, using (5.22) and (5.17), we get, for r sufficiently large (not depending 
on t), 

P(£(r,t)) < (1 + 2n r )K l e- K2rH + 2(n r ) 2 K 3 e~ K * sr 
< 9 1 (rH 2 + l){e~ e ^ 2t + r -°*W°), 

for some positive constants 6i, i = 1,2, 3. This completes the proof of Theo- 
rem 4.8. □ 

Proof of Theorem 4.9. Let do = d£o- We need to prove that, for 
sufficiently large r (not depending on t), 



(5.24) 



/ , . I {Q5( s )> d oiogr} dF 2 (s) / 

Jy3,r z t) 

< 7l(l + r 2 t)e^ 2rH + 73(1 + rHfr-^ . 
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Fix n > 1. Note that, from (5.4), it follows that, for s G [^2n-i; T 2n)' either 
Ql(s) > L r - 1 + j^Q\{s) > 0, or Ql(a) > C r - 1 > 0, for r large enough. 
From the form of the control policy, we have that the idle-time process for 
the second server, I 2 (-), does not increase during those intervals, and so 
the integrals over those intervals are zero. Thus, we need to consider only 
intervals of the form [T2 n _ 2 , T 2n-i)- We subdivide such intervals using a new 
sequence of stopping times as follows: 



(5.25) 



Vo 


— T 2n- 


2) 






~r,n 
V2m-1 


= min 


T2 n _i,inf{s > fj^_ 2 \Q r 3 (s) > (C r - 


-1)}], 


m = 1,2 


~r,n 
V2m 


= min 


r r 2n _ 1 M{s>vZ-i\Ql^)<(C r - 


-1)}], 


m = 1,2 


ar,n 
Pm 


= min 


~r,n 2 + 










mf| S >^_ 2 |Q^( S )>^logr} 




m= 1,2 



Next, we estimate, exactly as in the proof of Theorem 4.8, how many such 
sub-intervals [%m-2'%m-i) there can be within [0, r 2 t). Let n r be as in 
(5.7). Then from (5.8), we have that the probability in (5.24) is bounded by 



K±e 

(5.26) 



-i LJ\tZ_ „,t„_ ^ArH) 



n=l 



'2n-2>'2n- 



Now within these intervals, consider the subintervals formed by tf' n, s. By 
the form of the policy, Q r 3 (s) > C r — 1, for s £ [^m-n^m)- Thus, Q^(s) = 
is possible only for s E [%m-2; %m-i)- Thus, we can conclude that 



2n-2' T 2n-l Ar2 *) m=l J Ra£-2 .£-1 Ar 2 t) 



m=l ' 

Next observe that on the set T?2m-2 < r 2n-i A 

(5.28) Q r 3 (& 2 ) > L r - 1 for all r^_ 2 > fjf. 

To see this, consider the case of n > 2, m = 1. Note that from (5.4), Qs(^2m-2 _ 
Q5(r 2 r n _2-) > min{L r ,C r - 1}. This implies that Q r 3 (Vo' n ) > L r - 1. For 
the case n > l,m > 2, and r sufficiently large, we have that Q r 2,{^2m-2~) — 
C r - 1 > L r , which means Q§(i££_ 3 ) > L r - 1. This proves (5.28). 

Now define m r = [r 2 t(fji 2 + !)] + !• Note that %m r -i < ( r 2n-i Ar 2 t) implies 
that the queue-length Q^-) has crossed the threshold C r — 1 from below at 
least m r times before the time r 2 t, and each such up-crossing requires service 
of at least one job from Buffer 2, implying S' 2 (r 2 t) > m r . Using Corollary 
5.3, we get 

(5.29) P(&-i < rl n _ x A rh) < P(^(r 2 i) > m r ) < fte"*' 3 *, 
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for all r > f 5 = 1), where /3 3 = 9l({/4}> 1) , /3 4 = ^({^l' where 

f"i(-)> ?i (•) ' (•) are as in Corollary 5.3. Now, using (5.29) and (5.27), we 
write each summand of the second term in (5.26) as 



„ I {Q5(s)>d iog r} dl r 2 (s) / 0, T r 2n _ 2 < r t 

[ T 2 r n-2' T 2n-l Ar2i ) 



(5.30) <fo e -forH + Y^Jj^ 
m =i *-J[fj?,; 



W2m-2'''2m-l Ar V 



r ?2m-2 r 2n-l An 



From (5.26) and definitions of n r ,m r , we get from (5.30) the following bound 
on the probability in (5.24): 



< K ie ~ K2r2t + n r (3 3 e-P 4r2t 

i — ^ — ^ \ I r~T,n ~r,n , r >,\ 
n =lm=l VJ W2m-2^2m-l Ar ') 



l {Q r 2 {s)>d logr} dl r 2 (s) / 0, 



%m-2 < T 2n-l Ar * • 



By definition of /% n , for s E [j££_ 2 ,/2& n ), we have that Q r 2 (s) < ^logr. 
Therefore, for such s, the integrand on the right-hand side of (5.31) is zero. 
Hence, we have that 

each summand in (5.31) is bounded by 

(5.32) 

Now we make the following selections: 



(5.33) 



£1 E 0, min 



1 4 4(/f 2 -/x 3 ) 

#3 ?2 

K = 2max{4, 16A 2 , 32/x 2 , 16/z 3 }, 

d- e ( K 

, 2d 



0*2 - A»3)/2y ' 
4, 



mm 



{4'32d}' 



where #3 is as in Theorem 4.8 and ? 2 = ?2({A 2 },£ = if) ls as i n Corollary 
5.3. 
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By Assumption 2.1 and choice of e±, we can find r± > 1, such that, for all 
r > ri, 

(5.34) ^ 2 -^-2e 1 >^ T ^, ^ 2 + £l <2 / u 2 , ^ + £l <2 / i 3 . 

Define, for a > 0, A r 2 {s) = + s) - A^) , S^s) = S r j (1J(P^) + s) - 

S r j (l?(fa n )),j = 2,3. Now define 



(5.35) 



sup 

8(c* logr)<s<(c* logr) 



< ei, for j = 2,3 



4 n = ^ n n{ff<CiA^}. 



Observing that {/3 m n } are stopping times with respect to the filtration gen- 
erated by the queue-length processes, using the strong Markov property of 
the Poisson processes, and using Corollary 5.2, we have, for some constant 
C\ and function C-2(-) (not depending on r), 

(5.36) P[{iST] < C ie -c 2 ( £ i0)( c *io g r)_ 

Using (5.36), we can write (5.32) as 

I{05(*)>dbtogr} dI l( s ) + Mm™ < ^n-l A r 2 t 



(5.37) 



<F[{ATn{ffi"<rLiAr 2 t}] 



+ 1 

" '[ffi",£_iAA) 
< f7 e -C!2(ei0)(c*logr) 



I{Q5(s)>d logr} ^2 ( s ) / 0) A 



(5.38) 



(5.39) 



[/3m n ,rj2m-l Ar2 *) 

m — '/2m-2i 



I{Q-(s)>d logr} ^ 2 ( s ) / O^m™ 



l {QZ(s)>d logr} dI T 2 {s) / 0,^ 



Now consider the event corresponding to the probability in (5.38): 



(5.40) \^>f?i n 



m— 2' 



/ „ _ r „ I {Q5( s )> d ologr} dI T 2 {s) / 0, ^ n I = B^l 
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We claim that, for large values of r, 



(5.41) 



sup 

0<s<(6»c* logr) 



+ S) 



■ logr 



> 



1 /da 



2 V 4 



log r ; B, 



< 



qr 



where q" = e = f A 1) +ft({A£}, e = f A 1) and ^ = min{^({^}, e = 

^ A 1), <a({A 2 }, e = 4p A 1)}, where ?i and ? 2 are as in Corollary 5.3. To see 
this, note that, on £>^ n , by definition of /3^ n (recall that on the set £>^ n , 
P r m < r 2 t), w e have QKffi?) > f logr. And in order for Q r 2 (^ n + s) to 
decrease by |(^logr) inside [0, (9c* logr)], we need the number of services 
from Buffer 2 (by Server 1) to be greater than or equal to ^(^-logr) in 
(6c* logr) time. On the other hand, since on the set 5^ n , using /3^ n > 77 2 ^_ 2 , 
we have Q 2 ((3^—) < ^l°g r ) we can conclude that Q 2 (/?m n ) can be at 
most f logr + 1. And Q r 2 (/3^ n + •) > §(f- logr) inside [0, (0c* logr)] im- 
plies the number of arrivals in Buffer 2 needs to be greater than or equal 
to ^(^logr) — 1 > |(^logr) in time (9c* logr), for r large. Thus, for r 
sufficiently large, we can bound the probability in (5.41) by 



(5.42) 



S r 2 ( 



■)c* logr) > 



1 



logr 



+ : 



A r 2 (9c* logr) > 



1 



logr 



Also note that, if e 2 = min{l, 4r} and e 2 = min{l, 4^}, by the choices made 
in (5.33), we have, for r large enough, 

9c (n 2 + e 2 ) < 0— (2/x 2 ) < < - ( — 

0c (A 2 + e 2 ) < 0— (2A 2 ) < < - 

Using this observation and Corollary 5.3, we get that the sum in (5.42) is 
bounded by 

P[S£(0c*logr) > (fi r 2 + e 2 )9c*logr]+F[A r 2 (9c* logr) > (A 2 + e' 2 )9c* logr] 

This completes the proof of the claim in (5.41). 

Now, using (5.41), for large values of r, we can bound (5.38) by 



~m-2, / _ rn I {Q r 2 (s)>d logr}dI 2 ( s )^0,A r r ^ 

[Prk ,V 2 m-l Ar *) 



+ 



ffC-i - (W- 0c* logr <^ logr, 
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(5.43) 



sup Q^ + s )<!(^logr 



/ . n 
in 



(5.44) 



vZ-l-P r m n -Oc* logr logr, 



•r, n 
in 



Now we get a bound on each of (5.43) and (5.44). For the event in (5.43), 
note that Q r 2 (P^ 1 + s) < | ( ^ log r) for s < 6c* log r , and within an additional 
^logr units of time, Q^s) becomes greater than or equal to do log r [see 
the definition of the set in (5.40)]. This implies that there are more 
than (^logr) arrivals in Buffer 2 in time ^logr. Recalling the definition 
of e' 2 = min{l,A2/2} and the choices made in (5.33), we have that, for r 
large enough, 

(^2 + e 2 )^ < ( 2A 2J^ < -j- 

Observing that {P^ 1 } are stopping times with respect to the filtration gen- 
erated by the queue-length processes, using the strong Markov property of 
the Poisson processes, we get that the distribution (conditioned on the a- 
field generated by queue-length processes stopped at /3,J^ n ) of A^iOc* logr + 
•) — A^idc* logr) is the same as that of A^-). This, together with the display 
above, yields that, for r sufficiently large, (5.43) is bounded by 



Al 



do . 

— logr 



> 



— ) logr 



< 



AS I ^ Id. 



K 

-<$do/K 



>(A r 2 + 4)f logr 



where = ft({A2},£ = ^ A 1)}, i = 1,2, are as in Corollary 5.3. 

Next we show that, for r sufficiently large, (5.44) is zero. Note that by 
the choices made in (5.33), on A 7 ^-, we have that, for r sufficiently large, 



(5.45) 



S r 2 {s) < (// 2 + s 1 )s < (2/x 2 )^ logr < i logr 



for all s 6 



q * i 1 
ic log r, — - log r 

K 



Since on the set in (5.44) Q\ (P^ n + 6c* log r ) > \ ( f log r ) , this means Q\ (f3^ n + 
s) never becomes zero for s in the interval [6c* logr, (2do/K) logr]. So, on 
the set in (5.44), Q^iPl^ 1 + s) never becomes zero for s in the interval 
[0, (2do/K) logr]. Hence, using the fact that our policy requires Server 1 to 
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work on Buffer 2 continuously in the interval ffim + @ c * 1°S r + x r l > 
on the set in (5.44), we have that 

do 



+ Oc* log r + -^ logr 
> ,§2 f 0c* log r + ^ log r^j — S3 ^#c* log r + ^ log r 
(5 ' 46) > (^ - // 3 - 2ei) (flc* logr + ^ logr 

. (M2 ~ Ma) ^0 

>^^^logr 
= Co logr > C r . 

However, (5.46) is a contradiction to the fact that, on the set in (5.44), we 
have r]2m-\ > + ® c * l°g r + k log 7 "- This proves that (5.44) is zero. Thus, 
the term (5.38) is bounded by 

(5.47) +<&r-4 d °/ K . 

Now we consider (5.39). First note that for n = l,m = 1, (5.39) is zero, since 
(5^(0) = 0. For all other n > l,m > 1, consider the event corresponding to 
the probability in (5.39): 



6' 

I'll 



~r,n 

r ?2m-2' 



/ r oT.n ~r,n A o.\ 
'[Pm ,V 2 m-l Ar *) 

We claim that, for large values of r 



{Q£(s)>d logr 



inf 



+ s)< -t Q logr 



(5.48) 



U inf Q5(/% n + «) 

lo<s<(e»c*logr) 

<pir"^ c *, 



< 



1 fdn 



2 V 4 



logr 



m 



where pi = ?1 ({^},e = !f Al)+a({/i^},e = ^ Al) and p 2 = min{c 2 ({/4}, e 
4r A 1), ?2({^2}) e = ^ 1)} an d ft, « = 1,2, are as in Corollary 5.3. 



To see the claim, note that, on C^ n , 



r ?2m-2 



and from (5.28), Q§(t££_ 2 ) > 



^ologr — 1. And if r is large enough so that ^(^ologr) — 1 > -^logr, then 
in order for Q^iP^ 1 + •) to decrease by i(^ologr) — 1 in (9c* logr) time, 
we need the number of Class 3 services in that time interval to be greater 
than or equal to (^-logr). On the other hand, note that, by definition of 

PJ^ n , we have Q^Wm) > ^logr. And in order for Q r 2 (fi T m + to decrease 
by i(sjaiogr) inside [0, (6c* logr)], we need the number of Class 2 services 
in that time interval to be greater than or equal to i(^-logr). So for large 
values of r, the probability in (5.48) is bounded by 

1 fdo" 



(5.49) 



Sl( 



?c* logr) 



> — log r 

-4 



+ 



S£(0c*logr)> 



2 V 4 



logr 
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Also note that, by the choices made in (5.33), if we set £3 = min{l, /X3/2}, £4 = 
min{l, H2/2}, we have, for r large enough, that 

a *( r , \ ^ /j 2c ^o (n x 2fi 2 do 1 /do 
#c (/i 2 + £4) < 9—{2n2) < —j^— < - ( — 

Using the above observations and Corollary 5.3, we get that, for large r, the 
sum in (5.49) is bounded by 

¥[S r 3 (9c*logr) > (^ + £ 3 )6>c*logr] 

+ F[S r 2 {9c*logr) > {n r 2 + e 4 )6c*logr}< Pl r- p2dc * . 

This completes the proof of the claim (5.48). Now using (5.48), for large 
values of r, we can bound (5.39) as follows: 



r,n _ ~r,n 
m - 



< PIT 



I{Q^(s)>d logr} dl 2 (s) ^ 0,^ n 



(5.50) 



If d 



2cq 



(1*2 ~ to) 



log r, 



QW£ + s) > - If logr ) for a < 9c* logr, 
inf Q5(/^« + S )>^logr,C; 



+ : 



0c* log r> 



2c 



0*2 - /^3) 



log r, 



(5.51) 



Q r 2 (/?r + *)> 



1 fdn 



2 V 4 



log r I for s < #c* log r, 



inf QSGa£» + S )>-£logr,C 

0<s<»c *logr Z 



J7I 



We will next show that both the terms (5.50) and (5.51) are zero. First 
observe that, from (5.45), we have that, on A^ 1 , for r sufficiently large, 
Q^iPm 1 + ') never becomes zero in the interval [9c* logr, (2do/K) logr]. Thus, 
on the sets corresponding to terms (5.50) and (5.51), Q^O^m" + ') ^ s n °t zero 
on [0, {2d Q /K)\ogr\. 

On the event in (5.50), we have 



QWS + 9c*\ogr)>f\ogr 
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and 

[(^ + Oc*logr),fjZ-i] 



c 



(0^ + 9c* log r ) , + 0c* log r ) + - 2c ° log r 



Thus, by definition of C^ n and conditions of the event in (5.50), we must 
have that Q 3 (f3^ + 9c* logr + s) is zero for some s in [0, logr]. 
This means that, for some s in the above interval, [^(/^m™ + c* logr) — 
QzWm 1 + 0c* logr + s)] > ^logr. Now since Q 2 (P^ + •) never becomes 
zero in the interval [0, (2do/K) logr], this decrease in Q 3 (s) is bounded by 
[S%((9c* logr) + s) - S T 2 ((9c* log r) + s)] - [^(flc* logr) - S?;(0c* logr)]. Hence, 
the probability in (5.50) is bounded above by 

sup (S^(9c*logr + s) 

0<s<2c logr-/(^ 2 -A t 3) 

-S r 2 (9c*logr + s)) - (S r 3 (9c* logr) - S r 2 (0c* logr)) > ^ logr; .4^ 

We claim that, for all r large enough, the above probability is zero. To see the 
claim, note that, for all s < fj^zj^ , we have from (5.33) that 6c* logr + s£ 
[8c* log r, c* log r] . Thus, by definition of A r ^ and (5.33), we get, for all such 
s, that 

(S r 3 (9c* log r + s) - S r 2 (9c* log r + s) ) - (S r 3 (9c* log r ) - S r 2 (9c* log r ) ) 
< (^ - + 2ei) (0c* log r + a) - - ^ - 2ei ) (9c* log r ) 
= 4ei (0c* log r ) + (^ - ^ + 2 £l )s 
<4ei(0c* logr)-i(/i 2 -/x 3 )s 

<(4ei- |(/i 2 -n 3 ))(ec* logr) 
<0, 

for r large enough so that (fi 3 — pL 2 + 2e{) < — (fj, 2 — Hz)/2. This proves the 
claim. Thus, the expression in (5.50) is zero. We now show that (5.51) is 
zero as well. To see that, recall that, on the event in (5.51), Q 2 (P^ + •) 
never becomes zero in the interval [0, (2do /K) log r] . This implies that 

2cq log r 



(5.52) 



Q£ #>" + 0c*logr + 

(M2 - M3j. 

^arf Q *i . 2c logr \ ~ / 2c logr 
> A 2 I " c l°g r + 7 r -J3 fc log r + 



(A*2 — A*s)/ V (a*2 - M3) 

> (A*2 — A* 3 — 2e: l) [ 9c logr + 



0*2 - ^3 

. (A*2- A*3j 2c logr 

> z t -=c logr>C . 

2 (/i 2 - 1*3) 
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However, (5.52) contradicts the definition of %m-i m view of the fact that, 
on the set in (5.51), Tfem-i > + e c * lo S r + S=gj - This P roves that ( 5 - 51 ) 
is zero. Hence, we have proved that (5.39) is bounded by pir~ p2dc * . Using 
the above observation and (5.47), the term in (5.37) is bounded by 



A(r) = Cie" [c,2(eie)](c * logr:) + q[r~^ ec * + ^ r -^l K + pir ~ 



Using (5.32) and (5.31), we get the following bound on the left-hand side of 
(5.24), for large enough r: 



(5.53) 



[o,t) 



I {QSW>dologr/r} dr 2 (s)^0 



< K ie ~ K2T 1 + n r p 3 e~ 04r * + n r m r A(r) 

< 7i (1 + r 2 t)e-^ rH + 73 (1 + r 2 tfr-^\ 



for some constants 7, > 0, i = 1, . . . , 4, which are independent of t. This com- 
pletes the proof of the theorem. □ 



PROOF of Theorem 4.11. We will only prove (4.41). The proof of 
(4.40) is similar and therefore is omitted. Consider first the case i = 1. In 
view of (4.45), the main step is to obtain bounds for the following integrals: 



(5.54) 



sup |X[(s) 

0<s<t 



dt, 



1,2. 



By definition of -XJ(-), we have that, for i = 1,2, 
sup(A7( S )) 2 <3( SU p \Al(s)\) 2 

(5.55) 



0<s<t 



\0<s<t 

+ 3? sup |S[( S )|V + 3 (W4--); 



0<s<t 



By Doob's maximal inequality [for the martingale (A^(s) — A£s)], we have 



(5.56) 



<4A' 



sup |A[(s)| 

0<s<t 
00 

e 



2i 



dt 



/°°e-T*E[{ sup \Al(s)-X r iS \V 

JT L lo<s<r 2 i J 



T 



In a similar way, one shows that 



(5.57) 



sup |5[(s) 

0<s<t 



roo 

dt<4fj% J te'^dt. 
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Combining (5.56), (5.57), (5.55) and using Assumption 2.2, we obtain 



(5.58) limsuplimsup / e~ 7 *E 

T^oo r^oo Jt 



sup \X-(s) 

0<s<t 



2-i 



dt = 0, i = 1,2. 



Finally, combining (5.58) and (4.45) with the fact that T(-) is Lipschitz 
continuous, we have (4.41) for i = l. 

Proof of (4.41), for 2 = 2, is similar. We will only prove the key steps. Let 



(5.59) 



[o,t) 



L {Q r 2 (s)>d \ogr/r} 



o K r-M^2(s) 



From (4.49), it is clear that we need to get an estimate on 



(5.60) 



T 



e-^E{{Y t r } 2 ) dt 



T 



7i / F(Y t r > yfii)dudt. 



Theorem 4.9 and the fact that I^is) < rs yields the following bound on the 
integral in (5.60): 



F(Y t r >^I)du 



{Q£W>dologr/r}^2(s) > V^j du 



(5.61) 



I 

[0,t) 

[o,t) IlV2 

< r 2 t 2 ( 7l (l + r 2 t)e-^ rH + 73 (1 + r 2 tfr-^°). 



<r 2 t 2 



L {Q r 2 (s)>d logr/r} dI 2~( S )^° 



Substituting the above estimate in (5.60), one obtains after some straight- 
forward calculations that 

POO 

(5.62) limsuplimsup/ e~' yt E(Y t r ) 2 dt = 0. 

T^oo r-^oo JT 

Using (4.54), it follows that 



(5.63) limsuplimsup / e 7 *E sup 

T-+00 r^co Jt \0<s< 



Q r 2 (s) 



,r 1 {Q r 2 (s)<2d \ogr/r} 



dt = 0. 



s<t A* 3 

Now, as in the first half of the proof [see (5.58)], we can prove that 



/•OO 

(5.64) limsuplimsup / e~ 7< E 

T^oo r->co Jt 



sup \Xl{s) 

0<s<t 



2i 



dt = 0, z = 2,3. 



Thus, from (4.49), (5.62), (5.63) and the Lipschitz property of the Skorohod 
map, we get (4.41) for i = 2. This completes the proof of the theorem. □ 
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APPENDIX: PROOFS OF LEMMAS 4.5 AND 4.6 



Proof of Lemma 4.5. From (3.3), it follows that 
(A.l) (^(.),5 r (-))^(A(0,M0) asr^oo, 

where X(t) = Xt,fi(-) = [it;t > 0. Also, it follows from (2.7) and definition of 
the fluid-scaled processes in (2.8) that T r (-) is uniformly Lipschitz contin- 
uous with Lipschitz constant less than or equal to 1. This fact and (A.l) 
imply that 

(A.2) {^O,S r (0,T r OWisC-tight. 

Now, by definition of the queue-length process (2.1) and fluid-scaled pro- 
cesses (2.8), we have 

Ql(t) = A{(t)-S r 1 (n(t)), 
(A.3) Ql(t) = S r 2 (fr(t))-Sm(t)), 

ii{t)=t-fiit)-mi 

r 2 (t) = t-n(t). 

From (A.2), (A.3) and Lemma 3.14.1 of [2], we get 
(A.4) {Q r (-),I r (-)} r& isC-tight. 

Combining (A.2) and (A.4), we have (4.4). □ 

Proof of Lemma 4.6. From Lemma 4.5, we have 

(A.5) {</'(•), A r '(-),S r \.),f r \.),I r '(.)} r ^ 1 is C-tight. 

Thus, it is sufficient to show that all weak limit-points of the above sequence 
are given by the right-hand side of (4.7). 

Suppose that (Q(-),A(-),S(-),T(-),I(-)) is a limit-point of the sequence 
in (A.5), obtained along a subsequence indexed by r" . Using the Skorohod 
representation theorem, we can assume that this convergence takes place 
almost surely, uniformly on compacts: 

fA6 x (Q r "(-),A r "(.),S r "(-),f r "(.),F"(.)) 

From (A.l), we have that A(-) = A(-) and S(-) = //(•)• Recall that, by as- 
sumption, linv'—^oo J r (T r ) = J({T r }) < oo. Thus, using Fatou's lemma, 
we get 

0= lim — J r "(T r ")>E / limmf (h-Q r "{t))dt 

r"~^oo r \Jo r"^oo 

= E (jf°° e-T(h-Q(t))dt 
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Since hi > 0, i = 1,2,3, and Q has continuous paths, a.s., we have from the 
above equation that Q(-) = 0. Using this along with (A. 3) and (A.l), we now 
see that, for all £ > 0, 

= Ait-/iifi(t), = X 2 t-n 2 f 2 (t), = fi 2 f 2 (t)- /i 3 T 3 (t), 
h(t) =t - T x {t) - f 2 (t), I 2 (t) = t-f 3 (t). 

The result now follows on recalling the definition of T*(-) and Assumption 
2.2. □ 
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